Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-04-30T19:58:26Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-04-30T19:58:26Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260430 19:58:26.297464 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.62:33781
--webserver_interface=127.3.190.62
--webserver_port=0
--builtin_ntp_servers=127.3.190.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260430 19:58:26.588833 3841 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:58:26.589438 3841 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:58:26.589862 3841 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:58:26.622851 3841 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:58:26.623173 3841 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:58:26.623416 3841 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:58:26.623664 3841 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:58:26.660300 3841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.62:33781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.190.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:58:26.661659 3841 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:58:26.663372 3841 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:58:26.674898 3847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:26.675505 3848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:26.678740 3850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:27.795559 3849 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1116 milliseconds
I20260430 19:58:27.795675 3841 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:58:27.796983 3841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:58:27.799479 3841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:58:27.800853 3841 hybrid_clock.cc:648] HybridClock initialized: now 1777579107800794 us; error 68 us; skew 500 ppm
I20260430 19:58:27.801684 3841 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:58:27.808975 3841 webserver.cc:492] Webserver started at http://127.3.190.62:38679/ using document root <none> and password file <none>
I20260430 19:58:27.809988 3841 fs_manager.cc:362] Metadata directory not provided
I20260430 19:58:27.810241 3841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:58:27.810760 3841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:58:27.815719 3841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/instance:
uuid: "6b0a440dabed4814aaefe431944a7b5e"
format_stamp: "Formatted at 2026-04-30 19:58:27 on dist-test-slave-6zcr"
I20260430 19:58:27.816812 3841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal/instance:
uuid: "6b0a440dabed4814aaefe431944a7b5e"
format_stamp: "Formatted at 2026-04-30 19:58:27 on dist-test-slave-6zcr"
I20260430 19:58:27.823948 3841 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260430 19:58:27.829763 3857 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:27.830966 3841 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260430 19:58:27.831329 3841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
uuid: "6b0a440dabed4814aaefe431944a7b5e"
format_stamp: "Formatted at 2026-04-30 19:58:27 on dist-test-slave-6zcr"
I20260430 19:58:27.831660 3841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:58:27.883157 3841 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:58:27.884718 3841 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:58:27.885149 3841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:58:27.956571 3841 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.62:33781
I20260430 19:58:27.956617 3908 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.62:33781 every 8 connection(s)
I20260430 19:58:27.959283 3841 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
I20260430 19:58:27.965349 3909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:27.969066 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 3841
I20260430 19:58:27.969588 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal/instance
I20260430 19:58:27.975572 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.61:35559
--webserver_interface=127.3.190.61
--webserver_port=0
--builtin_ntp_servers=127.3.190.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260430 19:58:27.981025 3909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:58:28.005033 3909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } has no permanent_uuid. Determining permanent_uuid...
W20260430 19:58:28.007422 3861 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
W20260430 19:58:28.011427 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35559: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
I20260430 19:58:28.060998 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } attempt: 1
W20260430 19:58:28.065244 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35559: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
I20260430 19:58:28.133726 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } attempt: 2
W20260430 19:58:28.137559 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35559: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
I20260430 19:58:28.229107 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } attempt: 3
W20260430 19:58:28.233243 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35559: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
W20260430 19:58:28.289670 3911 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:58:28.290235 3911 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:58:28.290624 3911 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:58:28.324255 3911 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:58:28.324542 3911 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:58:28.324743 3911 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:58:28.324939 3911 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:58:28.362269 3911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.61:35559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.190.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:58:28.363696 3911 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:58:28.365453 3911 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260430 19:58:28.376709 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } attempt: 4
W20260430 19:58:28.377970 3920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:28.382342 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35559: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
I20260430 19:58:28.681804 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } attempt: 5
W20260430 19:58:28.688737 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35559: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
I20260430 19:58:29.236249 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } attempt: 6
W20260430 19:58:29.267249 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35559: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35559: connect: Connection refused (error 111)
W20260430 19:58:29.781312 3919 debug-util.cc:398] Leaking SignalData structure 0x7b0800034ca0 after lost signal to thread 3911
W20260430 19:58:29.888368 3919 kernel_stack_watchdog.cc:198] Thread 3911 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 399ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260430 19:58:28.379602 3921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:29.889880 3911 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.510s user 0.635s sys 0.874s
W20260430 19:58:29.890292 3911 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.511s user 0.635s sys 0.874s
W20260430 19:58:29.890424 3922 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1509 milliseconds
W20260430 19:58:29.892510 3923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:29.892441 3911 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:58:29.896566 3911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:58:29.898936 3911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:58:29.900309 3911 hybrid_clock.cc:648] HybridClock initialized: now 1777579109900289 us; error 63 us; skew 500 ppm
I20260430 19:58:29.901106 3911 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:58:29.907058 3911 webserver.cc:492] Webserver started at http://127.3.190.61:41103/ using document root <none> and password file <none>
I20260430 19:58:29.907966 3911 fs_manager.cc:362] Metadata directory not provided
I20260430 19:58:29.908185 3911 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:58:29.908622 3911 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:58:29.912849 3911 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data/instance:
uuid: "caa5357d0506482ea383b464e24f2a7a"
format_stamp: "Formatted at 2026-04-30 19:58:29 on dist-test-slave-6zcr"
I20260430 19:58:29.913929 3911 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/wal/instance:
uuid: "caa5357d0506482ea383b464e24f2a7a"
format_stamp: "Formatted at 2026-04-30 19:58:29 on dist-test-slave-6zcr"
I20260430 19:58:29.921044 3911 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260430 19:58:29.926918 3932 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:29.928028 3911 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260430 19:58:29.928406 3911 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/wal
uuid: "caa5357d0506482ea383b464e24f2a7a"
format_stamp: "Formatted at 2026-04-30 19:58:29 on dist-test-slave-6zcr"
I20260430 19:58:29.928743 3911 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:58:29.982231 3911 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:58:29.983764 3911 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:58:29.984220 3911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:58:30.054973 3911 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.61:35559
I20260430 19:58:30.055053 3983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.61:35559 every 8 connection(s)
I20260430 19:58:30.057678 3911 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
I20260430 19:58:30.063396 3984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:30.067340 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 3911
I20260430 19:58:30.067763 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-1/wal/instance
I20260430 19:58:30.069947 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.60:42577
--webserver_interface=127.3.190.60
--webserver_port=0
--builtin_ntp_servers=127.3.190.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260430 19:58:30.074344 3984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:58:30.100744 3984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:58:30.113929 3984 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } has no permanent_uuid. Determining permanent_uuid...
W20260430 19:58:30.116140 3935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
W20260430 19:58:30.120141 3984 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:30.169718 3984 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 1
W20260430 19:58:30.174028 3984 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:30.242453 3984 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 2
W20260430 19:58:30.246559 3984 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:30.327744 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } attempt: 7
I20260430 19:58:30.337987 3984 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 3
I20260430 19:58:30.339716 3909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } has no permanent_uuid. Determining permanent_uuid...
W20260430 19:58:30.342311 3984 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
W20260430 19:58:30.345536 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
W20260430 19:58:30.375276 3986 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:58:30.375896 3986 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:58:30.376490 3986 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260430 19:58:30.403944 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 1
W20260430 19:58:30.408411 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
W20260430 19:58:30.410262 3986 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:58:30.410565 3986 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:58:30.410856 3986 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:58:30.411083 3986 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:58:30.447583 3986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.60:42577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.190.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:58:30.448899 3986 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:58:30.450513 3986 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:58:30.461920 3999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:30.485792 3984 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 4
I20260430 19:58:30.489801 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 2
W20260430 19:58:30.493268 3984 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
W20260430 19:58:30.495610 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:30.581130 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 3
W20260430 19:58:30.586179 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:30.726712 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 4
W20260430 19:58:30.731890 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:30.792753 3984 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 5
W20260430 19:58:30.797855 3984 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:31.015385 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 5
W20260430 19:58:31.020447 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:31.345355 3984 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 6
W20260430 19:58:31.357102 3984 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
I20260430 19:58:31.572961 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 6
W20260430 19:58:31.580302 3909 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:42577: Network error: Client connection negotiation failed: client connection to 127.3.190.60:42577: connect: Connection refused (error 111)
W20260430 19:58:30.462304 3998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:31.587296 4001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:31.590972 4000 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1127 milliseconds
I20260430 19:58:31.591070 3986 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:58:31.592314 3986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:58:31.595104 3986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:58:31.596513 3986 hybrid_clock.cc:648] HybridClock initialized: now 1777579111596434 us; error 80 us; skew 500 ppm
I20260430 19:58:31.597411 3986 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:58:31.603706 3986 webserver.cc:492] Webserver started at http://127.3.190.60:36677/ using document root <none> and password file <none>
I20260430 19:58:31.604627 3986 fs_manager.cc:362] Metadata directory not provided
I20260430 19:58:31.604856 3986 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:58:31.605296 3986 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:58:31.611395 3986 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data/instance:
uuid: "dbc6c3c322c24063ad0cf6b243cf4755"
format_stamp: "Formatted at 2026-04-30 19:58:31 on dist-test-slave-6zcr"
I20260430 19:58:31.612475 3986 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/wal/instance:
uuid: "dbc6c3c322c24063ad0cf6b243cf4755"
format_stamp: "Formatted at 2026-04-30 19:58:31 on dist-test-slave-6zcr"
I20260430 19:58:31.619552 3986 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260430 19:58:31.625231 4012 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:31.626520 3986 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260430 19:58:31.626859 3986 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/wal
uuid: "dbc6c3c322c24063ad0cf6b243cf4755"
format_stamp: "Formatted at 2026-04-30 19:58:31 on dist-test-slave-6zcr"
I20260430 19:58:31.627167 3986 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:58:31.676484 3986 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:58:31.677914 3986 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:58:31.678305 3986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:58:31.744712 3986 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.60:42577
I20260430 19:58:31.744836 4063 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.60:42577 every 8 connection(s)
I20260430 19:58:31.747380 3986 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
I20260430 19:58:31.753000 4064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:31.753712 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 3986
I20260430 19:58:31.754093 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-2/wal/instance
I20260430 19:58:31.766809 4064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:58:31.791086 4064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:58:31.803040 4064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:58:31.820420 4064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Bootstrap starting.
I20260430 19:58:31.826508 4064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:31.828325 4064 log.cc:826] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Log is configured to *not* fsync() on all Append() calls
I20260430 19:58:31.833884 4064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: No bootstrap required, opened a new log
I20260430 19:58:31.854065 4064 raft_consensus.cc:359] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:31.854735 4064 raft_consensus.cc:385] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:31.854964 4064 raft_consensus.cc:740] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc6c3c322c24063ad0cf6b243cf4755, State: Initialized, Role: FOLLOWER
I20260430 19:58:31.855700 4064 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:31.857919 4075 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:31.858740 4075 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:31.859386 4064 sys_catalog.cc:565] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: configured and running, proceeding with master startup.
W20260430 19:58:31.882915 4086 catalog_manager.cc:1568] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:58:31.883180 4086 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:58:31.878865 3986 master_runner.cc:186] Error getting master registration for 127.3.190.62:33781: OK, instance_id { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" instance_seqno: 1777579107936303 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260430 19:58:31.904242 3986 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 19:58:31.933240 4018 tablet.cc:2404] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:58:32.130821 4075 raft_consensus.cc:493] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:32.131292 4075 raft_consensus.cc:515] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:32.133221 4075 leader_election.cc:290] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559)
I20260430 19:58:32.417613 3984 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 7
I20260430 19:58:32.437278 3984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: Bootstrap starting.
I20260430 19:58:32.445938 3984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:32.448712 3984 log.cc:826] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: Log is configured to *not* fsync() on all Append() calls
I20260430 19:58:32.453461 3984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: No bootstrap required, opened a new log
I20260430 19:58:32.480497 3984 raft_consensus.cc:359] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:32.481431 3984 raft_consensus.cc:385] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:32.481801 3984 raft_consensus.cc:740] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa5357d0506482ea383b464e24f2a7a, State: Initialized, Role: FOLLOWER
I20260430 19:58:32.482848 3984 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:32.485895 4091 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.486685 4091 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:32.487638 3984 sys_catalog.cc:565] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: configured and running, proceeding with master startup.
I20260430 19:58:32.487876 3959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:32.488715 3959 raft_consensus.cc:2468] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbc6c3c322c24063ad0cf6b243cf4755 in term 0.
I20260430 19:58:32.490130 4016 leader_election.cc:304] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755; no voters:
I20260430 19:58:32.491178 4075 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260430 19:58:32.491442 3938 tablet.cc:2404] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:58:32.491540 4075 raft_consensus.cc:493] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:32.491866 4075 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:32.499027 4075 raft_consensus.cc:515] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:32.503062 3959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a"
I20260430 19:58:32.503607 3959 raft_consensus.cc:3060] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:32.502758 4075 leader_election.cc:290] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 1 election: Requested vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559)
I20260430 19:58:32.511348 3959 raft_consensus.cc:2468] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc6c3c322c24063ad0cf6b243cf4755 in term 1.
I20260430 19:58:32.512532 4016 leader_election.cc:304] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755; no voters:
I20260430 19:58:32.513386 4075 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:32.520678 4075 raft_consensus.cc:697] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 1 LEADER]: Becoming Leader. State: Replica: dbc6c3c322c24063ad0cf6b243cf4755, State: Running, Role: LEADER
I20260430 19:58:32.521979 4075 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:32.537997 4097 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc6c3c322c24063ad0cf6b243cf4755. Latest consensus state: current_term: 1 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.539636 4097 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:32.543112 4104 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:58:32.550870 4104 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20260430 19:58:32.569950 4105 catalog_manager.cc:1568] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:58:32.570412 4105 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:58:32.586311 3959 raft_consensus.cc:1275] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Refusing update from remote peer dbc6c3c322c24063ad0cf6b243cf4755: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:32.588176 4097 consensus_queue.cc:1048] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:32.614674 3909 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } attempt: 7
I20260430 19:58:32.554667 3911 master_runner.cc:186] Error getting master registration for 127.3.190.62:33781: OK, instance_id { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" instance_seqno: 1777579107936303 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260430 19:58:32.621008 3911 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:58:32.659946 3909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Bootstrap starting.
I20260430 19:58:32.674554 4091 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc6c3c322c24063ad0cf6b243cf4755. Latest consensus state: current_term: 1 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.674877 4107 mvcc.cc:204] Tried to move back new op lower bound from 7280964045133275136 to 7280964044899893248. Current Snapshot: MvccSnapshot[applied={T|T < 7280964045133275136}]
I20260430 19:58:32.675637 4091 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:32.678864 4108 mvcc.cc:204] Tried to move back new op lower bound from 7280964045133275136 to 7280964044899893248. Current Snapshot: MvccSnapshot[applied={T|T < 7280964045133275136}]
I20260430 19:58:32.682915 4091 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.684933 4091 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:32.686347 3909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:32.691124 3909 log.cc:826] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Log is configured to *not* fsync() on all Append() calls
I20260430 19:58:32.695700 4097 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.697134 4097 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:32.699317 4104 catalog_manager.cc:1357] Generated new cluster ID: 56a2bf65e645471990682d6d15252142
I20260430 19:58:32.699625 4104 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
W20260430 19:58:32.705485 4015 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.190.62:33781 timed out after 0.571s (SENT)
I20260430 19:58:32.705858 3909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: No bootstrap required, opened a new log
I20260430 19:58:32.740658 3909 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:32.741647 3909 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:32.742004 3909 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0a440dabed4814aaefe431944a7b5e, State: Initialized, Role: FOLLOWER
I20260430 19:58:32.743103 3909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:32.758720 4104 catalog_manager.cc:1380] Generated new certificate authority record
I20260430 19:58:32.760823 4104 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:58:32.773974 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e"
I20260430 19:58:32.774699 3884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:32.775399 3883 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:32.775671 3884 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36390 (request call id 2) took 641 ms (client timeout 571 ms). Trace:
W20260430 19:58:32.776038 3884 rpcz_store.cc:269] 0430 19:58:32.133935 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:32.134105 (+ 170us) service_pool.cc:225] Handling call
0430 19:58:32.775586 (+641481us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260430 19:58:32.774034 3881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:32.782624 3883 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc6c3c322c24063ad0cf6b243cf4755 in term 1.
I20260430 19:58:32.783759 3882 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Refusing update from remote peer dbc6c3c322c24063ad0cf6b243cf4755: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:32.789983 4075 consensus_queue.cc:1048] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:32.790688 3909 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: configured and running, proceeding with master startup.
I20260430 19:58:32.790289 4115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.791100 4115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:32.803011 4104 catalog_manager.cc:6044] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Generated new TSK 0
I20260430 19:58:32.804086 4104 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:58:32.866070 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.190.1:0
--local_ip_for_outbound_sockets=127.3.190.1
--webserver_interface=127.3.190.1
--webserver_port=0
--tserver_master_addrs=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--builtin_ntp_servers=127.3.190.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260430 19:58:32.890544 4115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc6c3c322c24063ad0cf6b243cf4755. Latest consensus state: current_term: 1 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.891808 4115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:32.891544 4075 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.892207 4075 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:32.891131 4123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:32.893141 4123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:32.915509 4132 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260430 19:58:33.323408 4137 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:33.324188 4137 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:33.327435 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:33.330674 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:33.332494 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a; no voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755
W20260430 19:58:33.336609 4126 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:58:33.337316 4126 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260430 19:58:33.337525 4137 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
W20260430 19:58:33.338229 4126 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260430 19:58:33.339109 4137 raft_consensus.cc:2749] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260430 19:58:33.398001 4126 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:58:33.399626 4126 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.190.1
I20260430 19:58:33.412508 4139 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:33.413244 4139 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:33.416743 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:33.419905 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:33.421384 3860 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:33.427591 4139 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:33.428624 4139 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260430 19:58:33.465108 4126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.190.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.190.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.190.1
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:58:33.466850 4126 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:58:33.468978 4126 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:58:33.484531 4142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:33.576205 4105 catalog_manager.cc:1269] Loaded cluster ID: 56a2bf65e645471990682d6d15252142
I20260430 19:58:33.576610 4105 catalog_manager.cc:1562] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: loading cluster ID for follower catalog manager: success
I20260430 19:58:33.584100 4105 catalog_manager.cc:1584] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: acquiring CA information for follower catalog manager: success
I20260430 19:58:33.596092 4105 catalog_manager.cc:1612] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:58:33.936486 4132 catalog_manager.cc:1269] Loaded cluster ID: 56a2bf65e645471990682d6d15252142
I20260430 19:58:33.937784 4132 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: loading cluster ID for follower catalog manager: success
I20260430 19:58:33.997860 4149 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:33.998361 4149 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:34.000994 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:34.007181 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:34.008862 3860 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:34.005885 4149 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:34.014740 4150 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260430 19:58:34.016495 4132 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: acquiring CA information for follower catalog manager: success
I20260430 19:58:34.021970 4132 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:58:35.249804 4151 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:35.256743 4151 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:35.260607 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:35.266441 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:35.269423 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a; no voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:35.274155 4151 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:35.275470 4152 raft_consensus.cc:2749] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260430 19:58:34.884966 4141 debug-util.cc:398] Leaking SignalData structure 0x7b0800028ca0 after lost signal to thread 4126
W20260430 19:58:35.683168 4141 kernel_stack_watchdog.cc:198] Thread 4126 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 396ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260430 19:58:33.488319 4143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:35.686650 4145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:35.685336 4126 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 2.198s user 0.507s sys 0.771s
W20260430 19:58:35.687376 4126 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 2.201s user 0.507s sys 0.772s
W20260430 19:58:35.689013 4144 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 2201 milliseconds
I20260430 19:58:35.689119 4126 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:58:35.690655 4126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:58:35.693396 4126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:58:35.694849 4126 hybrid_clock.cc:648] HybridClock initialized: now 1777579115694734 us; error 108 us; skew 500 ppm
I20260430 19:58:35.696020 4126 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:58:35.705423 4126 webserver.cc:492] Webserver started at http://127.3.190.1:43589/ using document root <none> and password file <none>
I20260430 19:58:35.706748 4126 fs_manager.cc:362] Metadata directory not provided
I20260430 19:58:35.707037 4126 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:58:35.707612 4126 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:58:35.714039 4126 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data/instance:
uuid: "5a7356e99e744525861e12c108cfa3fe"
format_stamp: "Formatted at 2026-04-30 19:58:35 on dist-test-slave-6zcr"
I20260430 19:58:35.715754 4126 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/instance:
uuid: "5a7356e99e744525861e12c108cfa3fe"
format_stamp: "Formatted at 2026-04-30 19:58:35 on dist-test-slave-6zcr"
I20260430 19:58:35.725618 4126 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260430 19:58:35.733444 4157 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:35.734911 4126 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.004s
I20260430 19:58:35.735327 4126 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal
uuid: "5a7356e99e744525861e12c108cfa3fe"
format_stamp: "Formatted at 2026-04-30 19:58:35 on dist-test-slave-6zcr"
I20260430 19:58:35.735749 4126 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:58:35.780642 4151 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:35.781140 4151 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:35.783394 4151 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:35.785583 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:35.786908 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:35.789041 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a; no voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:35.794749 4151 raft_consensus.cc:2749] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260430 19:58:35.826543 4126 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:58:35.828526 4126 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:58:35.829082 4126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:58:35.834972 4126 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260430 19:58:35.841112 4126 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260430 19:58:35.841418 4126 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:35.841720 4126 ts_tablet_manager.cc:616] Registered 0 tablets
I20260430 19:58:35.841943 4126 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:36.050836 4126 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.1:33065
I20260430 19:58:36.050935 4269 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.1:33065 every 8 connection(s)
I20260430 19:58:36.053436 4126 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/data/info.pb
I20260430 19:58:36.058799 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 4126
I20260430 19:58:36.059394 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/instance
I20260430 19:58:36.097733 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.190.2:0
--local_ip_for_outbound_sockets=127.3.190.2
--webserver_interface=127.3.190.2
--webserver_port=0
--tserver_master_addrs=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--builtin_ntp_servers=127.3.190.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260430 19:58:36.117815 4271 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35559
I20260430 19:58:36.118361 4271 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:36.119617 4271 heartbeater.cc:507] Master 127.3.190.61:35559 requested a full tablet report, sending...
I20260430 19:58:36.124114 3949 ts_manager.cc:194] Registered new tserver with Master: 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:36.124328 4270 heartbeater.cc:344] Connected to a master server at 127.3.190.60:42577
I20260430 19:58:36.124656 4270 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:36.126075 4270 heartbeater.cc:507] Master 127.3.190.60:42577 requested a full tablet report, sending...
I20260430 19:58:36.128588 4029 ts_manager.cc:194] Registered new tserver with Master: 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:36.130913 4274 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:36.131786 4274 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:36.133409 4274 heartbeater.cc:507] Master 127.3.190.62:33781 requested a full tablet report, sending...
I20260430 19:58:36.134343 4029 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.190.1:59387
I20260430 19:58:36.136484 3874 ts_manager.cc:194] Registered new tserver with Master: 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:36.335196 4282 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:36.335716 4282 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:36.337801 4282 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:36.338685 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:36.339322 3958 raft_consensus.cc:2468] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0a440dabed4814aaefe431944a7b5e in term 1.
I20260430 19:58:36.339409 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:36.340760 3861 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e, caa5357d0506482ea383b464e24f2a7a; no voters:
I20260430 19:58:36.345091 4282 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260430 19:58:36.345567 4282 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Starting leader election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:36.345978 4282 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 1 FOLLOWER]: Advancing to term 2
I20260430 19:58:36.377110 4282 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:36.380676 4282 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 election: Requested vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:36.380867 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a"
I20260430 19:58:36.382441 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755"
I20260430 19:58:36.383793 3860 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:36.384567 4282 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260430 19:58:36.460352 4280 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:58:36.460912 4280 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:58:36.461468 4280 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:58:36.495862 4280 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:58:36.496764 4280 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.190.2
I20260430 19:58:36.548931 4280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.190.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.190.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.190.2
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:58:36.550381 4280 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:58:36.552042 4280 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:58:36.574558 4290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:36.638896 4270 heartbeater.cc:499] Master 127.3.190.60:42577 was elected leader, sending a full tablet report...
I20260430 19:58:36.801465 3883 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Rejecting Update request from peer dbc6c3c322c24063ad0cf6b243cf4755 for earlier term 1. Current term is 2. Ops: []
I20260430 19:58:36.808069 4284 consensus_queue.cc:1059] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260430 19:58:36.809257 4284 raft_consensus.cc:3055] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 1 LEADER]: Stepping down as leader of term 1
I20260430 19:58:36.809684 4284 raft_consensus.cc:740] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dbc6c3c322c24063ad0cf6b243cf4755, State: Running, Role: LEADER
I20260430 19:58:36.811107 4284 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:36.812770 4284 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 1 FOLLOWER]: Advancing to term 2
I20260430 19:58:36.864034 4296 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:36.865823 4296 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:36.869540 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:36.870939 3883 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate caa5357d0506482ea383b464e24f2a7a in current term 2: Already voted for candidate 6b0a440dabed4814aaefe431944a7b5e in this term.
I20260430 19:58:36.874462 4296 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:36.874573 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:36.875259 4039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caa5357d0506482ea383b464e24f2a7a in term 2.
I20260430 19:58:36.876461 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755; no voters: 6b0a440dabed4814aaefe431944a7b5e
I20260430 19:58:36.877525 4296 raft_consensus.cc:2804] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260430 19:58:36.877871 4296 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Starting leader election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:36.878372 4296 raft_consensus.cc:3060] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 1 FOLLOWER]: Advancing to term 2
I20260430 19:58:36.884806 4296 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:36.887420 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e"
I20260430 19:58:36.888159 3883 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate caa5357d0506482ea383b464e24f2a7a in current term 2: Already voted for candidate 6b0a440dabed4814aaefe431944a7b5e in this term.
I20260430 19:58:36.891657 4086 catalog_manager.cc:1612] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:58:36.891786 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755"
I20260430 19:58:36.898965 4039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caa5357d0506482ea383b464e24f2a7a in term 2.
I20260430 19:58:36.900206 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755; no voters: 6b0a440dabed4814aaefe431944a7b5e
I20260430 19:58:36.905519 4296 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 2 election: Requested vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:36.906577 4296 raft_consensus.cc:2804] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 2 FOLLOWER]: Leader election won for term 2
I20260430 19:58:36.907704 4296 raft_consensus.cc:697] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 2 LEADER]: Becoming Leader. State: Replica: caa5357d0506482ea383b464e24f2a7a, State: Running, Role: LEADER
I20260430 19:58:36.909263 4296 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:36.940351 4297 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa5357d0506482ea383b464e24f2a7a. Latest consensus state: current_term: 2 leader_uuid: "caa5357d0506482ea383b464e24f2a7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:36.941241 4297 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:36.954186 4299 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:58:36.963184 4299 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:58:36.967661 4299 catalog_manager.cc:1269] Loaded cluster ID: 56a2bf65e645471990682d6d15252142
I20260430 19:58:36.967986 4299 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:58:36.973270 4299 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:58:36.978000 4299 catalog_manager.cc:6055] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: Loaded TSK: 0
I20260430 19:58:36.979512 4299 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:58:37.137986 4300 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:37.138860 4300 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:37.143810 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:37.154282 4300 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:37.155586 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:37.156162 4039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0a440dabed4814aaefe431944a7b5e in term 2.
I20260430 19:58:37.157562 3860 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755; no voters: caa5357d0506482ea383b464e24f2a7a
I20260430 19:58:37.159034 4300 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260430 19:58:37.159536 4300 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:37.159905 4300 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 2 FOLLOWER]: Advancing to term 3
I20260430 19:58:37.167542 4300 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:37.170148 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a"
I20260430 19:58:37.172917 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755"
I20260430 19:58:37.173427 4039 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 2 FOLLOWER]: Advancing to term 3
I20260430 19:58:37.179659 4039 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0a440dabed4814aaefe431944a7b5e in term 3.
I20260430 19:58:37.180801 3860 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755; no voters: caa5357d0506482ea383b464e24f2a7a
I20260430 19:58:37.186437 4300 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 3 election: Requested vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:37.186934 4300 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 3 FOLLOWER]: Leader election won for term 3
I20260430 19:58:37.187837 4300 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 3 LEADER]: Becoming Leader. State: Replica: 6b0a440dabed4814aaefe431944a7b5e, State: Running, Role: LEADER
I20260430 19:58:37.188807 4300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:37.194787 4301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0a440dabed4814aaefe431944a7b5e. Latest consensus state: current_term: 3 leader_uuid: "6b0a440dabed4814aaefe431944a7b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:37.195322 4301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:37.196715 4303 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:58:37.202025 4303 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:58:37.204707 4303 catalog_manager.cc:1269] Loaded cluster ID: 56a2bf65e645471990682d6d15252142
I20260430 19:58:37.204923 4303 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:58:37.208568 4303 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:58:37.211261 4303 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Loaded TSK: 0
I20260430 19:58:37.212225 4303 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:58:37.425760 4039 raft_consensus.cc:1240] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Rejecting Update request from peer caa5357d0506482ea383b464e24f2a7a for earlier term 2. Current term is 3. Ops: []
I20260430 19:58:37.427637 4297 consensus_queue.cc:1059] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260430 19:58:37.428634 4296 raft_consensus.cc:3055] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 2 LEADER]: Stepping down as leader of term 2
I20260430 19:58:37.429003 4296 raft_consensus.cc:740] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: caa5357d0506482ea383b464e24f2a7a, State: Running, Role: LEADER
I20260430 19:58:37.429809 4296 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:37.431093 4296 raft_consensus.cc:3060] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 2 FOLLOWER]: Advancing to term 3
W20260430 19:58:37.570044 3905 debug-util.cc:398] Leaking SignalData structure 0x7b08000a61e0 after lost signal to thread 3842
W20260430 19:58:37.571081 3905 debug-util.cc:398] Leaking SignalData structure 0x7b080006f840 after lost signal to thread 3908
I20260430 19:58:37.611914 3958 raft_consensus.cc:1275] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Refusing update from remote peer 6b0a440dabed4814aaefe431944a7b5e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260430 19:58:37.612293 3958 pending_rounds.cc:85] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: Aborting all ops after (but not including) 4
I20260430 19:58:37.612583 3958 pending_rounds.cc:107] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260430 19:58:37.612836 3958 raft_consensus.cc:2889] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260430 19:58:37.615520 4300 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260430 19:58:37.622148 4296 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0a440dabed4814aaefe431944a7b5e. Latest consensus state: current_term: 3 leader_uuid: "6b0a440dabed4814aaefe431944a7b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:37.622917 4296 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:37.638821 4271 heartbeater.cc:499] Master 127.3.190.61:35559 was elected leader, sending a full tablet report...
I20260430 19:58:37.657363 4300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6b0a440dabed4814aaefe431944a7b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:37.658058 4300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:37.659930 4039 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Refusing update from remote peer 6b0a440dabed4814aaefe431944a7b5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260430 19:58:37.661934 4296 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6b0a440dabed4814aaefe431944a7b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:37.662581 4296 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:37.667521 4301 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260430 19:58:37.707026 4307 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0a440dabed4814aaefe431944a7b5e. Latest consensus state: current_term: 3 leader_uuid: "6b0a440dabed4814aaefe431944a7b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:37.707798 4307 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:37.711078 4310 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6b0a440dabed4814aaefe431944a7b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:37.712551 4310 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: FOLLOWER
W20260430 19:58:36.575843 4292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:37.923820 4294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:37.926000 4280 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.349s user 0.005s sys 0.004s
W20260430 19:58:37.926254 4280 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.349s user 0.005s sys 0.004s
I20260430 19:58:37.926513 4280 server_base.cc:1061] running on GCE node
I20260430 19:58:37.927536 4280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:58:37.929845 4280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:58:37.931335 4280 hybrid_clock.cc:648] HybridClock initialized: now 1777579117931293 us; error 43 us; skew 500 ppm
I20260430 19:58:37.932171 4280 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:58:37.941120 4280 webserver.cc:492] Webserver started at http://127.3.190.2:40889/ using document root <none> and password file <none>
I20260430 19:58:37.942068 4280 fs_manager.cc:362] Metadata directory not provided
I20260430 19:58:37.942283 4280 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:58:37.942791 4280 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:58:37.947678 4280 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data/instance:
uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
format_stamp: "Formatted at 2026-04-30 19:58:37 on dist-test-slave-6zcr"
I20260430 19:58:37.948819 4280 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/instance:
uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
format_stamp: "Formatted at 2026-04-30 19:58:37 on dist-test-slave-6zcr"
I20260430 19:58:37.956146 4280 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260430 19:58:37.962886 4316 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:37.964303 4280 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260430 19:58:37.964712 4280 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal
uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
format_stamp: "Formatted at 2026-04-30 19:58:37 on dist-test-slave-6zcr"
I20260430 19:58:37.965088 4280 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:58:38.044039 4280 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:58:38.045702 4280 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:58:38.046175 4280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:58:38.067217 4280 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260430 19:58:38.075398 4280 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260430 19:58:38.075712 4280 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:38.076007 4280 ts_tablet_manager.cc:616] Registered 0 tablets
I20260430 19:58:38.076208 4280 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:38.169641 4274 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:38.181960 4310 raft_consensus.cc:493] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:38.182503 4310 raft_consensus.cc:515] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:38.184263 4310 leader_election.cc:290] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559)
I20260430 19:58:38.185765 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:38.186043 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:38.187477 4015 leader_election.cc:304] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbc6c3c322c24063ad0cf6b243cf4755; no voters: 6b0a440dabed4814aaefe431944a7b5e, caa5357d0506482ea383b464e24f2a7a
I20260430 19:58:38.188261 4310 raft_consensus.cc:2749] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260430 19:58:38.291893 4301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6b0a440dabed4814aaefe431944a7b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:38.292701 4301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:38.338658 4280 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.2:43939
I20260430 19:58:38.338778 4433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.2:43939 every 8 connection(s)
I20260430 19:58:38.341912 4280 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/data/info.pb
I20260430 19:58:38.348469 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 4280
I20260430 19:58:38.349222 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/instance
I20260430 19:58:38.373119 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.190.3:0
--local_ip_for_outbound_sockets=127.3.190.3
--webserver_interface=127.3.190.3
--webserver_port=0
--tserver_master_addrs=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--builtin_ntp_servers=127.3.190.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260430 19:58:38.406927 4434 heartbeater.cc:344] Connected to a master server at 127.3.190.60:42577
I20260430 19:58:38.407439 4434 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:38.408733 4434 heartbeater.cc:507] Master 127.3.190.60:42577 requested a full tablet report, sending...
I20260430 19:58:38.411890 4029 ts_manager.cc:194] Registered new tserver with Master: d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:38.423362 4436 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:38.423751 4436 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:38.423846 4435 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35559
I20260430 19:58:38.424208 4435 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:38.424651 4436 heartbeater.cc:507] Master 127.3.190.62:33781 requested a full tablet report, sending...
I20260430 19:58:38.425078 4435 heartbeater.cc:507] Master 127.3.190.61:35559 requested a full tablet report, sending...
I20260430 19:58:38.427042 3874 ts_manager.cc:194] Registered new tserver with Master: d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:38.427589 3949 ts_manager.cc:194] Registered new tserver with Master: d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:38.429711 3874 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.190.2:52675
W20260430 19:58:38.717376 4444 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:58:38.717876 4444 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:58:38.718557 4444 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:58:38.759227 4444 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:58:38.760898 4444 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.190.3
I20260430 19:58:38.793607 4449 raft_consensus.cc:493] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:38.794094 4449 raft_consensus.cc:515] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:38.795907 4449 leader_election.cc:290] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559)
I20260430 19:58:38.797070 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:38.797227 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:38.798830 4016 leader_election.cc:304] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbc6c3c322c24063ad0cf6b243cf4755; no voters: 6b0a440dabed4814aaefe431944a7b5e, caa5357d0506482ea383b464e24f2a7a
I20260430 19:58:38.799564 4449 raft_consensus.cc:2749] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260430 19:58:38.813442 4444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.190.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.190.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.190.3
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:58:38.814841 4444 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:58:38.816394 4444 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:58:38.829425 4451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:38.933844 4436 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:39.004673 4457 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:39.005540 4457 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:39.008752 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:39.010053 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:39.012351 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a; no voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:39.017850 4457 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:39.019052 4457 raft_consensus.cc:2749] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260430 19:58:39.287863 4266 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad1c0 after lost signal to thread 4133
W20260430 19:58:39.289111 4266 debug-util.cc:398] Leaking SignalData structure 0x7b08000adf60 after lost signal to thread 4269
I20260430 19:58:39.787366 4459 raft_consensus.cc:493] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:39.788807 4459 raft_consensus.cc:515] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:39.791926 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:39.803138 4459 leader_election.cc:290] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559)
I20260430 19:58:39.804261 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:39.806013 4016 leader_election.cc:304] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbc6c3c322c24063ad0cf6b243cf4755; no voters: 6b0a440dabed4814aaefe431944a7b5e, caa5357d0506482ea383b464e24f2a7a
I20260430 19:58:39.810814 4459 raft_consensus.cc:2749] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260430 19:58:40.000679 4460 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:40.001315 4460 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:40.004388 3883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6b0a440dabed4814aaefe431944a7b5e" is_pre_election: true
I20260430 19:58:40.020124 4460 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:40.020402 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:40.022202 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a; no voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:40.023378 4460 raft_consensus.cc:2749] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260430 19:58:38.831662 4452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:38.833797 4454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:58:40.414215 4453 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1574 milliseconds
I20260430 19:58:40.414369 4444 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:58:40.415561 4444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:58:40.418016 4444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:58:40.419446 4444 hybrid_clock.cc:648] HybridClock initialized: now 1777579120419406 us; error 58 us; skew 500 ppm
I20260430 19:58:40.420248 4444 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:58:40.426963 4444 webserver.cc:492] Webserver started at http://127.3.190.3:38545/ using document root <none> and password file <none>
I20260430 19:58:40.427871 4444 fs_manager.cc:362] Metadata directory not provided
I20260430 19:58:40.428076 4444 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:58:40.428540 4444 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:58:40.433549 4444 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data/instance:
uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
format_stamp: "Formatted at 2026-04-30 19:58:40 on dist-test-slave-6zcr"
I20260430 19:58:40.434643 4444 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/instance:
uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
format_stamp: "Formatted at 2026-04-30 19:58:40 on dist-test-slave-6zcr"
I20260430 19:58:40.442044 4444 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260430 19:58:40.447858 4466 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:40.448989 4444 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20260430 19:58:40.449311 4444 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal
uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
format_stamp: "Formatted at 2026-04-30 19:58:40 on dist-test-slave-6zcr"
I20260430 19:58:40.449630 4444 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:58:40.506397 4444 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:58:40.507905 4444 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:58:40.508351 4444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:58:40.511252 4444 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260430 19:58:40.515983 4444 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260430 19:58:40.516202 4444 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:40.516454 4444 ts_tablet_manager.cc:616] Registered 0 tablets
I20260430 19:58:40.516603 4444 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:40.689249 4444 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.3:37953
I20260430 19:58:40.689433 4578 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.3:37953 every 8 connection(s)
I20260430 19:58:40.692404 4444 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/data/info.pb
I20260430 19:58:40.697505 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 4444
I20260430 19:58:40.697964 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/instance
I20260430 19:58:40.767570 4579 heartbeater.cc:344] Connected to a master server at 127.3.190.60:42577
I20260430 19:58:40.768277 4579 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:40.769755 4579 heartbeater.cc:507] Master 127.3.190.60:42577 requested a full tablet report, sending...
I20260430 19:58:40.774438 4029 ts_manager.cc:194] Registered new tserver with Master: 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:40.789403 4580 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35559
I20260430 19:58:40.789672 4581 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:40.789848 4580 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:40.789992 4581 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:40.790916 4581 heartbeater.cc:507] Master 127.3.190.62:33781 requested a full tablet report, sending...
I20260430 19:58:40.791033 4580 heartbeater.cc:507] Master 127.3.190.61:35559 requested a full tablet report, sending...
I20260430 19:58:40.793792 3874 ts_manager.cc:194] Registered new tserver with Master: 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:40.794186 3949 ts_manager.cc:194] Registered new tserver with Master: 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:40.795672 3874 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.190.3:55317
I20260430 19:58:40.813997 3832 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260430 19:58:40.888645 3873 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57350:
name: "default.Table_00566458e6454e9c9de18a15b89fd71a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:40.892113 3871 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57350:
name: "default.Table_83ba28cbb72d4c30ae7009ee9e022fd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:40.889422 3872 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57350:
name: "default.Table_e7d2ad532a85474a8f5a4d4a17aa44d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:40.923262 3872 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7d2ad532a85474a8f5a4d4a17aa44d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:40.923391 3871 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83ba28cbb72d4c30ae7009ee9e022fd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:40.923250 3873 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00566458e6454e9c9de18a15b89fd71a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:40.881742 3874 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57350:
name: "default.Table_cfd44e2b17f24afa88671a852e6fb260"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:40.929459 3874 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfd44e2b17f24afa88671a852e6fb260 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:40.934108 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 3841
W20260430 19:58:40.964958 4319 connection.cc:570] client connection to 127.3.190.62:33781 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:40.965988 4436 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:33781 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260430 19:58:41.098461 4614 raft_consensus.cc:493] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:41.099366 4614 raft_consensus.cc:515] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:41.105696 4614 leader_election.cc:290] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:41.111084 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "caa5357d0506482ea383b464e24f2a7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
W20260430 19:58:41.112627 3935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260430 19:58:41.118660 3935 leader_election.cc:336] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111)
I20260430 19:58:41.119102 3935 leader_election.cc:304] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a; no voters: 6b0a440dabed4814aaefe431944a7b5e, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:41.119843 4614 raft_consensus.cc:2749] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260430 19:58:41.200136 4274 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:33781 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111)
I20260430 19:58:41.251474 4618 raft_consensus.cc:493] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:41.252077 4618 raft_consensus.cc:515] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:41.254119 4618 leader_election.cc:290] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559)
I20260430 19:58:41.255537 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:41.256309 3958 raft_consensus.cc:2468] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbc6c3c322c24063ad0cf6b243cf4755 in term 3.
I20260430 19:58:41.257958 4016 leader_election.cc:304] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755; no voters:
I20260430 19:58:41.258952 4618 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260430 19:58:41.259333 4618 raft_consensus.cc:493] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b0a440dabed4814aaefe431944a7b5e)
I20260430 19:58:41.259724 4618 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 3 FOLLOWER]: Advancing to term 4
W20260430 19:58:41.262158 4015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111)
I20260430 19:58:41.268690 4618 raft_consensus.cc:515] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:41.271291 4618 leader_election.cc:290] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 election: Requested vote from peers 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781), caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559)
I20260430 19:58:41.272650 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a"
I20260430 19:58:41.273401 3958 raft_consensus.cc:3060] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 3 FOLLOWER]: Advancing to term 4
W20260430 19:58:41.275597 4015 leader_election.cc:336] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111)
W20260430 19:58:41.280579 4015 leader_election.cc:336] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111)
I20260430 19:58:41.287916 3958 raft_consensus.cc:2468] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbc6c3c322c24063ad0cf6b243cf4755 in term 4.
I20260430 19:58:41.289654 4016 leader_election.cc:304] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755; no voters: 6b0a440dabed4814aaefe431944a7b5e
I20260430 19:58:41.290766 4618 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 4 FOLLOWER]: Leader election won for term 4
I20260430 19:58:41.292150 4618 raft_consensus.cc:697] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [term 4 LEADER]: Becoming Leader. State: Replica: dbc6c3c322c24063ad0cf6b243cf4755, State: Running, Role: LEADER
I20260430 19:58:41.293053 4618 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:41.299728 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:41.307972 4621 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc6c3c322c24063ad0cf6b243cf4755. Latest consensus state: current_term: 4 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:41.309023 4621 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: LEADER
W20260430 19:58:41.313856 4581 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:33781 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111)
I20260430 19:58:41.315179 4624 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:58:41.323546 4624 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:58:41.327806 4624 catalog_manager.cc:1269] Loaded cluster ID: 56a2bf65e645471990682d6d15252142
I20260430 19:58:41.328074 4624 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:58:41.333153 4624 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:58:41.337298 4624 catalog_manager.cc:6055] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Loaded TSK: 0
I20260430 19:58:41.338522 4624 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20260430 19:58:41.774773 4015 consensus_peers.cc:597] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 -> Peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Couldn't send request to peer 6b0a440dabed4814aaefe431944a7b5e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:41.852808 3958 raft_consensus.cc:1275] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [term 4 FOLLOWER]: Refusing update from remote peer dbc6c3c322c24063ad0cf6b243cf4755: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260430 19:58:41.854007 4621 consensus_queue.cc:1048] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260430 19:58:41.859537 4628 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc6c3c322c24063ad0cf6b243cf4755. Latest consensus state: current_term: 4 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:41.860430 4628 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:41.864627 4621 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:41.865326 4621 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 [sys.catalog]: This master's current role is: LEADER
I20260430 19:58:41.866741 4628 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:41.867514 4628 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa5357d0506482ea383b464e24f2a7a [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:41.942214 4434 heartbeater.cc:499] Master 127.3.190.60:42577 was elected leader, sending a full tablet report...
W20260430 19:58:40.961614 4591 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.190.62:33781)
W20260430 19:58:42.160815 3837 debug-util.cc:398] Leaking SignalData structure 0x7b080002d0a0 after lost signal to thread 4591
W20260430 19:58:42.183133 4591 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.CreateTable -> {remote=127.3.190.62:33781, user_credentials={real_user=slave}} blocked reactor thread for 1.22264e+06us
W20260430 19:58:42.184827 4591 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.190.62:33781)
I20260430 19:58:42.185031 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.62:33781
--webserver_interface=127.3.190.62
--webserver_port=38679
--builtin_ntp_servers=127.3.190.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260430 19:58:42.203652 4270 heartbeater.cc:499] Master 127.3.190.60:42577 was elected leader, sending a full tablet report...
I20260430 19:58:42.202885 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_e7d2ad532a85474a8f5a4d4a17aa44d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:42.202780 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_83ba28cbb72d4c30ae7009ee9e022fd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:42.203851 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_00566458e6454e9c9de18a15b89fd71a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:42.205508 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_cfd44e2b17f24afa88671a852e6fb260"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:42.206496 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83ba28cbb72d4c30ae7009ee9e022fd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:42.207235 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfd44e2b17f24afa88671a852e6fb260 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:42.206323 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7d2ad532a85474a8f5a4d4a17aa44d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:42.211642 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00566458e6454e9c9de18a15b89fd71a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.244498 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_e7d2ad532a85474a8f5a4d4a17aa44d5" } new_table_name: "default.Table_cdb8e345d8054b74824b2fd4a92b10d3" modify_external_catalogs: true
I20260430 19:58:42.256739 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_00566458e6454e9c9de18a15b89fd71a" } new_table_name: "default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c" modify_external_catalogs: true
I20260430 19:58:42.265755 4029 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_83ba28cbb72d4c30ae7009ee9e022fd0" } modify_external_catalogs: true
I20260430 19:58:42.266363 4029 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_83ba28cbb72d4c30ae7009ee9e022fd0" } modify_external_catalogs: true
I20260430 19:58:42.269691 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cfd44e2b17f24afa88671a852e6fb260" } modify_external_catalogs: true
I20260430 19:58:42.270215 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cfd44e2b17f24afa88671a852e6fb260" } modify_external_catalogs: true
W20260430 19:58:42.283066 4027 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 79a11487b7734e66817a993b1434926e has no consensus state
I20260430 19:58:42.284722 4579 heartbeater.cc:499] Master 127.3.190.60:42577 was elected leader, sending a full tablet report...
W20260430 19:58:42.283340 4028 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a7a82e60c974997b52a26650b131fc0 consensus state has no leader
I20260430 19:58:42.314322 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7a82e60c974997b52a26650b131fc0 (DEFAULT_TABLE table=default.Table_e7d2ad532a85474a8f5a4d4a17aa44d5 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.316742 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7a82e60c974997b52a26650b131fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.328294 4029 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 3994f10a426d4302b5c3c7f9a9f70b9e
I20260430 19:58:42.329061 4026 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet d4757f1d8bde4336b84fdca64fafeb6a
W20260430 19:58:42.338445 4015 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 79a11487b7734e66817a993b1434926e (table default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c [id=94190b075a2a4a828681f5bb6594f8a2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 79a11487b7734e66817a993b1434926e has no consensus state
I20260430 19:58:42.345187 4364 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7a82e60c974997b52a26650b131fc0 (DEFAULT_TABLE table=default.Table_e7d2ad532a85474a8f5a4d4a17aa44d5 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.348196 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7a82e60c974997b52a26650b131fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.369082 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7a82e60c974997b52a26650b131fc0 (DEFAULT_TABLE table=default.Table_cdb8e345d8054b74824b2fd4a92b10d3 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.371299 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7a82e60c974997b52a26650b131fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.386116 4204 tablet_service.cc:1511] Processing CreateTablet for tablet 79a11487b7734e66817a993b1434926e (DEFAULT_TABLE table=default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.387415 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a11487b7734e66817a993b1434926e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.388248 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 79a11487b7734e66817a993b1434926e (DEFAULT_TABLE table=default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.389376 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a11487b7734e66817a993b1434926e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.391790 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 79a11487b7734e66817a993b1434926e (DEFAULT_TABLE table=default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.392968 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a11487b7734e66817a993b1434926e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.413024 4026 master_service.cc:954] ReplaceTablet: received request to replace tablet 79a11487b7734e66817a993b1434926e from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:42.421744 4029 master_service.cc:954] ReplaceTablet: received request to replace tablet 79a11487b7734e66817a993b1434926e from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:42.433445 4652 tablet_bootstrap.cc:492] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:42.445587 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 79a11487b7734e66817a993b1434926e
I20260430 19:58:42.447324 4026 catalog_manager.cc:6741] ReplaceTablet: tablet 79a11487b7734e66817a993b1434926e deleted and replaced by tablet 6856b41ce4ae450ca7b08015666c3a2d
I20260430 19:58:42.451399 4362 tablet_service.cc:1558] Processing DeleteTablet for tablet 79a11487b7734e66817a993b1434926e with delete_type TABLET_DATA_DELETED (replaced by tablet 6856b41ce4ae450ca7b08015666c3a2d) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:42.451359 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 79a11487b7734e66817a993b1434926e with delete_type TABLET_DATA_DELETED (replaced by tablet 6856b41ce4ae450ca7b08015666c3a2d) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:42.455358 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ec013029c0de4691aee33eec63af4b84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:42.457239 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec013029c0de4691aee33eec63af4b84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.461664 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_4a891ffb2bad4492b969437e45c01394"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:42.463716 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a891ffb2bad4492b969437e45c01394 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.471907 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_6385d4cb63ac46559cabe66291dbba05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:42.473608 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6385d4cb63ac46559cabe66291dbba05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.473723 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_70e56a97501b455da85a1a264e94ecd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:42.475739 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70e56a97501b455da85a1a264e94ecd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.486155 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 79a11487b7734e66817a993b1434926e with delete_type TABLET_DATA_DELETED (replaced by tablet 6856b41ce4ae450ca7b08015666c3a2d) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:42.489748 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d (DEFAULT_TABLE table=default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.489774 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d (DEFAULT_TABLE table=default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.490862 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6856b41ce4ae450ca7b08015666c3a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.493424 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6856b41ce4ae450ca7b08015666c3a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.493304 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d (DEFAULT_TABLE table=default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.494488 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6856b41ce4ae450ca7b08015666c3a2d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:42.498139 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 79a11487b7734e66817a993b1434926e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79a11487b7734e66817a993b1434926e already in progress: creating tablet
I20260430 19:58:42.500259 4024 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5fdfa7d1091a4d4fb5b2395e03bec61c" } new_table_name: "default.Table_86c5dceec81b4ab9817b32775f015c68" modify_external_catalogs: true
W20260430 19:58:42.501140 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 79a11487b7734e66817a993b1434926e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79a11487b7734e66817a993b1434926e already in progress: creating tablet
I20260430 19:58:42.504570 4026 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cdb8e345d8054b74824b2fd4a92b10d3" } new_table_name: "default.Table_872a450ea8d144c4a4c4d5cd12195d82" modify_external_catalogs: true
W20260430 19:58:42.505579 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 79a11487b7734e66817a993b1434926e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79a11487b7734e66817a993b1434926e already in progress: creating tablet
I20260430 19:58:42.550894 4653 tablet_bootstrap.cc:492] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:42.555557 4027 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4a891ffb2bad4492b969437e45c01394" } modify_external_catalogs: true
I20260430 19:58:42.556143 4027 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4a891ffb2bad4492b969437e45c01394" } modify_external_catalogs: true
I20260430 19:58:42.558928 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_6385d4cb63ac46559cabe66291dbba05" } modify_external_catalogs: true
I20260430 19:58:42.559466 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_6385d4cb63ac46559cabe66291dbba05" } modify_external_catalogs: true
I20260430 19:58:42.560624 4658 tablet_bootstrap.cc:492] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:42.562727 4652 tablet_bootstrap.cc:700] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent opening tablet: real 0.122s user 0.000s sys 0.003s
I20260430 19:58:42.563534 4652 tablet_bootstrap.cc:654] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:42.570899 4652 log.cc:826] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Log is configured to *not* fsync() on all Append() calls
I20260430 19:58:42.572857 4653 tablet_bootstrap.cc:654] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:42.575417 4653 log.cc:826] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Log is configured to *not* fsync() on all Append() calls
I20260430 19:58:42.577983 4658 tablet_bootstrap.cc:654] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:42.580343 4024 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6856b41ce4ae450ca7b08015666c3a2d consensus state has no leader
I20260430 19:58:42.580437 4658 log.cc:826] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Log is configured to *not* fsync() on all Append() calls
W20260430 19:58:42.592417 4026 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a7a82e60c974997b52a26650b131fc0 consensus state has no leader
I20260430 19:58:42.594353 4652 tablet_bootstrap.cc:492] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:42.594981 4652 ts_tablet_manager.cc:1403] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.162s user 0.010s sys 0.008s
I20260430 19:58:42.599753 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 (DEFAULT_TABLE table=default.Table_4a891ffb2bad4492b969437e45c01394 [id=9dd258a7ab884edca842031038fa5c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.600909 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a24d4962ee542e2a66c6e48b9a2b692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.603314 4364 tablet_service.cc:1511] Processing CreateTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 (DEFAULT_TABLE table=default.Table_4a891ffb2bad4492b969437e45c01394 [id=9dd258a7ab884edca842031038fa5c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.603394 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 (DEFAULT_TABLE table=default.Table_4a891ffb2bad4492b969437e45c01394 [id=9dd258a7ab884edca842031038fa5c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.604408 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a24d4962ee542e2a66c6e48b9a2b692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.604462 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a24d4962ee542e2a66c6e48b9a2b692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.622713 4652 raft_consensus.cc:359] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:42.623601 4652 raft_consensus.cc:385] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:42.623708 4028 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 2201f8bbdd0644b9bcbc6b59a5905b20
I20260430 19:58:42.623921 4652 raft_consensus.cc:740] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:42.624781 4652 consensus_queue.cc:260] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:42.631484 4658 tablet_bootstrap.cc:492] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:42.632086 4658 ts_tablet_manager.cc:1403] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.073s user 0.009s sys 0.007s
I20260430 19:58:42.637326 4027 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:42.642052 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:42 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:42.645298 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:42 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:42.650933 4027 master_service.cc:954] ReplaceTablet: received request to replace tablet 79a11487b7734e66817a993b1434926e from {username='slave'} at 127.0.0.1:55130
W20260430 19:58:42.652316 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 8a24d4962ee542e2a66c6e48b9a2b692 because the tablet was not found. No further retry: Not found: Tablet not found: 8a24d4962ee542e2a66c6e48b9a2b692
W20260430 19:58:42.653103 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 8a24d4962ee542e2a66c6e48b9a2b692 because the tablet was not found. No further retry: Not found: Tablet not found: 8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:42.657745 4653 tablet_bootstrap.cc:492] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:42.658320 4653 ts_tablet_manager.cc:1403] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.108s user 0.018s sys 0.007s
I20260430 19:58:42.660475 4027 master_service.cc:954] ReplaceTablet: received request to replace tablet 6856b41ce4ae450ca7b08015666c3a2d from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:42.666173 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:42 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:42.668660 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 8a24d4962ee542e2a66c6e48b9a2b692 because the tablet was not found. No further retry: Not found: Tablet not found: 8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:42.682636 4658 raft_consensus.cc:359] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.683564 4658 raft_consensus.cc:385] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:42.683871 4658 raft_consensus.cc:740] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:42.689617 4658 consensus_queue.cc:260] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.705024 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 (DEFAULT_TABLE table=default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.706115 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.705317 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 (DEFAULT_TABLE table=default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.707279 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.707705 4364 tablet_service.cc:1511] Processing CreateTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 (DEFAULT_TABLE table=default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.708783 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.710687 4205 tablet_service.cc:1511] Processing CreateTablet for tablet d0f228d11b9641589825bb6a61310672 (DEFAULT_TABLE table=default.Table_70e56a97501b455da85a1a264e94ecd8 [id=50df359f50de4b88a31deb7922615f33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.711722 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0f228d11b9641589825bb6a61310672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.715656 4363 tablet_service.cc:1511] Processing CreateTablet for tablet d0f228d11b9641589825bb6a61310672 (DEFAULT_TABLE table=default.Table_70e56a97501b455da85a1a264e94ecd8 [id=50df359f50de4b88a31deb7922615f33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.716667 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0f228d11b9641589825bb6a61310672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.722127 4514 tablet_service.cc:1511] Processing CreateTablet for tablet d0f228d11b9641589825bb6a61310672 (DEFAULT_TABLE table=default.Table_70e56a97501b455da85a1a264e94ecd8 [id=50df359f50de4b88a31deb7922615f33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.723183 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0f228d11b9641589825bb6a61310672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.735344 4027 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 6856b41ce4ae450ca7b08015666c3a2d
I20260430 19:58:42.736811 4027 catalog_manager.cc:6741] ReplaceTablet: tablet 6856b41ce4ae450ca7b08015666c3a2d deleted and replaced by tablet d1d52994ff694886b7d4ff4224367082
I20260430 19:58:42.739358 4204 tablet_service.cc:1558] Processing DeleteTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d with delete_type TABLET_DATA_DELETED (replaced by tablet d1d52994ff694886b7d4ff4224367082) from {username='slave'} at 127.0.0.1:41996
W20260430 19:58:42.740975 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 6856b41ce4ae450ca7b08015666c3a2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6856b41ce4ae450ca7b08015666c3a2d already in progress: creating tablet
I20260430 19:58:42.741542 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d with delete_type TABLET_DATA_DELETED (replaced by tablet d1d52994ff694886b7d4ff4224367082) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:42.742429 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d with delete_type TABLET_DATA_DELETED (replaced by tablet d1d52994ff694886b7d4ff4224367082) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:42.744032 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6856b41ce4ae450ca7b08015666c3a2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6856b41ce4ae450ca7b08015666c3a2d already in progress: creating tablet
I20260430 19:58:42.744482 4658 ts_tablet_manager.cc:1434] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.112s user 0.031s sys 0.007s
W20260430 19:58:42.744715 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 6856b41ce4ae450ca7b08015666c3a2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6856b41ce4ae450ca7b08015666c3a2d already in progress: creating tablet
I20260430 19:58:42.745515 4658 tablet_bootstrap.cc:492] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:42.748391 4652 ts_tablet_manager.cc:1434] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.153s user 0.040s sys 0.011s
I20260430 19:58:42.752147 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_21e25c2aeb1e4c4585e86a91709f1d52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:42.752579 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_65bbaf481196477c8b0ac8a125da24bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:42.753815 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21e25c2aeb1e4c4585e86a91709f1d52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:42.754177 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65bbaf481196477c8b0ac8a125da24bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.758453 4658 tablet_bootstrap.cc:654] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:42.764509 4635 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:58:42.765297 4635 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:58:42.766105 4635 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260430 19:58:42.766909 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet 79a11487b7734e66817a993b1434926e with delete_type TABLET_DATA_DELETED (replaced by tablet 6856b41ce4ae450ca7b08015666c3a2d) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:42.767802 4654 tablet_replica.cc:333] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:42.768610 4654 raft_consensus.cc:2243] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:42.769208 4654 raft_consensus.cc:2272] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:42.770920 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:42.772444 4654 ts_tablet_manager.cc:1916] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:42.772647 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.775725 4653 raft_consensus.cc:359] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:42.778648 4653 raft_consensus.cc:385] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:42.779438 4653 raft_consensus.cc:740] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:42.780951 4653 consensus_queue.cc:260] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:42.786942 4026 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_70e56a97501b455da85a1a264e94ecd8" } new_table_name: "default.Table_2cc5216057ba4cd688be5c6fbd4a6f69" modify_external_catalogs: true
I20260430 19:58:42.797928 4024 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_86c5dceec81b4ab9817b32775f015c68" } new_table_name: "default.Table_acca5aade33a4c9b923d16ee8e9ef5a8" modify_external_catalogs: true
I20260430 19:58:42.801658 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_78209fb594f841308f82995e31eff165"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:42.803282 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78209fb594f841308f82995e31eff165 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:42.804162 4652 tablet_bootstrap.cc:492] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:42.815541 4658 tablet_bootstrap.cc:492] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:42.815982 4658 ts_tablet_manager.cc:1403] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.071s user 0.017s sys 0.003s
I20260430 19:58:42.818827 4658 raft_consensus.cc:359] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.819566 4658 raft_consensus.cc:385] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:42.819886 4658 raft_consensus.cc:740] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:42.820636 4658 consensus_queue.cc:260] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.827044 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_65bbaf481196477c8b0ac8a125da24bb" } modify_external_catalogs: true
I20260430 19:58:42.827574 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_65bbaf481196477c8b0ac8a125da24bb" } modify_external_catalogs: true
I20260430 19:58:42.830235 4658 ts_tablet_manager.cc:1434] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260430 19:58:42.842937 4658 tablet_bootstrap.cc:492] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:42.844393 4652 tablet_bootstrap.cc:654] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:42.846271 4653 ts_tablet_manager.cc:1434] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.187s user 0.030s sys 0.011s
I20260430 19:58:42.849867 4653 tablet_bootstrap.cc:492] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
W20260430 19:58:42.854790 4635 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:58:42.855190 4635 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:58:42.855511 4635 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:58:42.855875 4635 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:58:42.855790 4654 ts_tablet_manager.cc:1929] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:42.859160 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_21e25c2aeb1e4c4585e86a91709f1d52" } modify_external_catalogs: true
I20260430 19:58:42.859690 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_21e25c2aeb1e4c4585e86a91709f1d52" } modify_external_catalogs: true
I20260430 19:58:42.863427 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet 8a24d4962ee542e2a66c6e48b9a2b692 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:42.870820 4653 tablet_bootstrap.cc:654] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:42.871268 4026 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0f228d11b9641589825bb6a61310672 consensus state has no leader
I20260430 19:58:42.873082 4652 tablet_bootstrap.cc:492] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:42.873548 4652 ts_tablet_manager.cc:1403] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.070s user 0.005s sys 0.007s
I20260430 19:58:42.877703 4652 raft_consensus.cc:359] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.882262 4652 raft_consensus.cc:385] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:42.882676 4652 raft_consensus.cc:740] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:42.882911 4653 tablet_bootstrap.cc:492] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:42.883423 4653 ts_tablet_manager.cc:1403] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.014s
I20260430 19:58:42.884012 4652 consensus_queue.cc:260] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:42.886991 4024 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d1d52994ff694886b7d4ff4224367082 has no consensus state
I20260430 19:58:42.886188 4653 raft_consensus.cc:359] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.888199 4652 ts_tablet_manager.cc:1434] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260430 19:58:42.888434 4653 raft_consensus.cc:385] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:42.888732 4653 raft_consensus.cc:740] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:42.889487 4653 consensus_queue.cc:260] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.891762 4653 ts_tablet_manager.cc:1434] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.008s user 0.003s sys 0.002s
I20260430 19:58:42.895315 4652 tablet_bootstrap.cc:492] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:42.898140 4653 tablet_bootstrap.cc:492] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:42.856185 4654 log.cc:1199] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/79a11487b7734e66817a993b1434926e
I20260430 19:58:42.904419 4653 tablet_bootstrap.cc:654] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:42.914363 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_872a450ea8d144c4a4c4d5cd12195d82 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 2 got 0
I20260430 19:58:42.916062 4513 tablet_service.cc:1511] Processing CreateTablet for tablet d1d52994ff694886b7d4ff4224367082 (DEFAULT_TABLE table=default.Table_acca5aade33a4c9b923d16ee8e9ef5a8 [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.917230 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1d52994ff694886b7d4ff4224367082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.919371 4652 tablet_bootstrap.cc:654] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:42.919481 4023 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_872a450ea8d144c4a4c4d5cd12195d82 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 2 got 0
I20260430 19:58:42.924186 4205 tablet_service.cc:1511] Processing CreateTablet for tablet d1d52994ff694886b7d4ff4224367082 (DEFAULT_TABLE table=default.Table_acca5aade33a4c9b923d16ee8e9ef5a8 [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.925374 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1d52994ff694886b7d4ff4224367082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.949369 4023 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_872a450ea8d144c4a4c4d5cd12195d82 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 2 got 0
I20260430 19:58:42.954144 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 79a11487b7734e66817a993b1434926e with delete_type TABLET_DATA_DELETED (replaced by tablet 6856b41ce4ae450ca7b08015666c3a2d) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:42.955060 4655 tablet_replica.cc:333] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:42.955633 4028 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 834ec7c41cc144c6894d9755e962cfe7
I20260430 19:58:42.955894 4655 raft_consensus.cc:2243] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:42.956434 4655 raft_consensus.cc:2272] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:42.960939 4652 tablet_bootstrap.cc:492] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:42.961355 4652 ts_tablet_manager.cc:1403] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.066s user 0.015s sys 0.007s
I20260430 19:58:42.961758 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_872a450ea8d144c4a4c4d5cd12195d82 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 2 got 0
I20260430 19:58:42.963143 4655 ts_tablet_manager.cc:1916] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:42.964524 4652 raft_consensus.cc:359] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.965180 4652 raft_consensus.cc:385] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:42.965679 4652 raft_consensus.cc:740] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:42.868360 4658 tablet_bootstrap.cc:654] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:42.966447 4652 consensus_queue.cc:260] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:42.893651 4364 tablet_service.cc:1558] Processing DeleteTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d with delete_type TABLET_DATA_DELETED (replaced by tablet d1d52994ff694886b7d4ff4224367082) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:42.926999 4439 tablet.cc:2404] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:58:42.930580 4363 tablet_service.cc:1511] Processing CreateTablet for tablet d1d52994ff694886b7d4ff4224367082 (DEFAULT_TABLE table=default.Table_acca5aade33a4c9b923d16ee8e9ef5a8 [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:42.969794 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1d52994ff694886b7d4ff4224367082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:42.967384 4654 ts_tablet_manager.cc:1950] T 79a11487b7734e66817a993b1434926e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:42.975448 4652 ts_tablet_manager.cc:1434] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20260430 19:58:42.976475 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 79a11487b7734e66817a993b1434926e with delete_type TABLET_DATA_DELETED (replaced by tablet 6856b41ce4ae450ca7b08015666c3a2d) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:42.977625 4657 tablet_replica.cc:333] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:42.978441 4657 raft_consensus.cc:2243] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:42.979003 4657 raft_consensus.cc:2272] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:42.982261 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:42 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:42.983351 4029 master_service.cc:954] ReplaceTablet: received request to replace tablet 8a24d4962ee542e2a66c6e48b9a2b692 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:42.983497 4657 ts_tablet_manager.cc:1916] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:42.990375 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 79a11487b7734e66817a993b1434926e (table default.Table_acca5aade33a4c9b923d16ee8e9ef5a8 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:42.991829 4654 tablet_replica.cc:333] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:42.996724 4022 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 2a4d87eb3e8a4e1b9bfe8b2b4975ac9a
I20260430 19:58:42.999212 4652 tablet_bootstrap.cc:492] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
W20260430 19:58:43.003176 4586 tablet.cc:2404] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:58:43.011070 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_e4bcca1db8ad434780968ac6fe236a2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:43.012696 4658 tablet_bootstrap.cc:492] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
W20260430 19:58:43.012822 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4bcca1db8ad434780968ac6fe236a2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.013159 4658 ts_tablet_manager.cc:1403] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.170s user 0.016s sys 0.000s
I20260430 19:58:43.014403 4652 tablet_bootstrap.cc:654] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.016659 4658 raft_consensus.cc:359] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:43.020493 4635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:33781,127.3.190.61:35559,127.3.190.60:42577
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.62:33781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.190.62
--webserver_port=38679
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:58:43.023765 4658 raft_consensus.cc:385] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.024525 4658 raft_consensus.cc:740] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.024704 4657 ts_tablet_manager.cc:1929] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.025445 4657 log.cc:1199] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/79a11487b7734e66817a993b1434926e
I20260430 19:58:43.025913 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_12e78c5061bc4945b51efe881ca1d3f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:43.026897 4662 raft_consensus.cc:493] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:43.027436 4662 raft_consensus.cc:515] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:43.025580 4658 consensus_queue.cc:260] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:43.028827 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ace9ebd59ec0464cbca1f928a48eedaa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:43.034179 4658 ts_tablet_manager.cc:1434] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.021s user 0.003s sys 0.003s
I20260430 19:58:43.036752 4655 ts_tablet_manager.cc:1929] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.037187 4655 log.cc:1199] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/79a11487b7734e66817a993b1434926e
I20260430 19:58:43.038061 4655 ts_tablet_manager.cc:1950] T 79a11487b7734e66817a993b1434926e P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:43.040251 4658 tablet_bootstrap.cc:492] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:43.046408 4662 leader_election.cc:290] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:43.047720 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 79a11487b7734e66817a993b1434926e (table default.Table_acca5aade33a4c9b923d16ee8e9ef5a8 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
W20260430 19:58:43.052670 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ace9ebd59ec0464cbca1f928a48eedaa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.055729 4658 tablet_bootstrap.cc:654] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:42.992573 4654 raft_consensus.cc:2243] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:43.056362 4654 raft_consensus.cc:2272] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.062191 4657 ts_tablet_manager.cc:1950] T 79a11487b7734e66817a993b1434926e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:43.066120 4654 ts_tablet_manager.cc:1916] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:43.066535 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12e78c5061bc4945b51efe881ca1d3f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.066839 4657 tablet_replica.cc:333] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:43.068089 4657 raft_consensus.cc:2243] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:43.068141 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 79a11487b7734e66817a993b1434926e (table default.Table_acca5aade33a4c9b923d16ee8e9ef5a8 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:43.068588 4657 raft_consensus.cc:2272] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.075489 4022 master_service.cc:954] ReplaceTablet: received request to replace tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:43.076408 4023 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_acca5aade33a4c9b923d16ee8e9ef5a8" } new_table_name: "default.Table_58ec6ea9c8dc4e2c9d35ab8648add502" modify_external_catalogs: true
I20260430 19:58:43.079214 4635 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:58:43.085472 4658 tablet_bootstrap.cc:492] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:43.085913 4658 ts_tablet_manager.cc:1403] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.003s
I20260430 19:58:43.088174 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 6108085995074ea890cc6933bf45661f (DEFAULT_TABLE table=default.Table_78209fb594f841308f82995e31eff165 [id=f509dee5b07f496381f8a0edcbb50ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:43.088583 4658 raft_consensus.cc:359] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.089229 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6108085995074ea890cc6933bf45661f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:43.092144 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 6108085995074ea890cc6933bf45661f (DEFAULT_TABLE table=default.Table_78209fb594f841308f82995e31eff165 [id=f509dee5b07f496381f8a0edcbb50ced]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:43.092613 4277 tablet.cc:2404] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:58:43.093286 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6108085995074ea890cc6933bf45661f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:43.094746 4652 tablet_bootstrap.cc:492] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:43.095166 4652 ts_tablet_manager.cc:1403] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.096s user 0.015s sys 0.000s
I20260430 19:58:43.095057 4025 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_872a450ea8d144c4a4c4d5cd12195d82" } new_table_name: "default.Table_d5d25ec643f44b2190b0473839ea4da1" modify_external_catalogs: true
I20260430 19:58:43.097575 4652 raft_consensus.cc:359] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.098222 4652 raft_consensus.cc:385] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.103893 4657 ts_tablet_manager.cc:1916] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:43.105621 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 6108085995074ea890cc6933bf45661f (DEFAULT_TABLE table=default.Table_78209fb594f841308f82995e31eff165 [id=f509dee5b07f496381f8a0edcbb50ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:43.106688 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6108085995074ea890cc6933bf45661f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:43.081413 4635 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260430 19:58:43.110998 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_5cbfe354833844588121cc44d0a76e64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:43.112771 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5cbfe354833844588121cc44d0a76e64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.089203 4658 raft_consensus.cc:385] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.117305 4658 raft_consensus.cc:740] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.117691 4024 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_872a450ea8d144c4a4c4d5cd12195d82 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 2 got 0
I20260430 19:58:43.117944 4658 consensus_queue.cc:260] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.121204 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7a82e60c974997b52a26650b131fc0" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:43.122157 4658 ts_tablet_manager.cc:1434] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20260430 19:58:43.122009 4225 raft_consensus.cc:2468] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:43.123055 4658 tablet_bootstrap.cc:492] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:43.124363 4470 leader_election.cc:304] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:43.125221 4662 raft_consensus.cc:2804] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:43.125561 4662 raft_consensus.cc:493] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:43.125865 4662 raft_consensus.cc:3060] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:43.133353 4662 raft_consensus.cc:515] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:43.134892 4658 tablet_bootstrap.cc:654] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.142091 4653 tablet_bootstrap.cc:492] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:43.098475 4652 raft_consensus.cc:740] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.142530 4653 ts_tablet_manager.cc:1403] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.245s user 0.034s sys 0.047s
I20260430 19:58:43.143580 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 504523705ff944c7b3f84db0ddf9f455 (DEFAULT_TABLE table=default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e [id=0f3d9fa7646043459be13fcce244dc0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:43.143381 4652 consensus_queue.cc:260] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.144680 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 504523705ff944c7b3f84db0ddf9f455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:43.145596 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7a82e60c974997b52a26650b131fc0" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:43.146101 4225 raft_consensus.cc:3060] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:43.146147 4653 raft_consensus.cc:359] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.146843 4653 raft_consensus.cc:385] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.147119 4653 raft_consensus.cc:740] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.147719 4653 consensus_queue.cc:260] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.152211 4653 ts_tablet_manager.cc:1434] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260430 19:58:43.157825 4653 tablet_bootstrap.cc:492] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:43.159168 4021 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_78209fb594f841308f82995e31eff165" } modify_external_catalogs: true
I20260430 19:58:43.159749 4021 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_78209fb594f841308f82995e31eff165" } modify_external_catalogs: true
I20260430 19:58:43.163422 4225 raft_consensus.cc:2468] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:43.164000 4662 leader_election.cc:290] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:43.164861 4653 tablet_bootstrap.cc:654] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.174540 4470 leader_election.cc:304] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:43.175544 4662 raft_consensus.cc:2804] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:43.176020 4662 raft_consensus.cc:697] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:43.178761 4662 consensus_queue.cc:237] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:43.186472 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 504523705ff944c7b3f84db0ddf9f455 (DEFAULT_TABLE table=default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e [id=0f3d9fa7646043459be13fcce244dc0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:43.187762 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 504523705ff944c7b3f84db0ddf9f455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:43.188246 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7a82e60c974997b52a26650b131fc0" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:43.189138 4384 raft_consensus.cc:2468] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:43.199735 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7a82e60c974997b52a26650b131fc0" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:43.200374 4383 raft_consensus.cc:3060] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:43.204010 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 504523705ff944c7b3f84db0ddf9f455 (DEFAULT_TABLE table=default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e [id=0f3d9fa7646043459be13fcce244dc0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:43.205139 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 504523705ff944c7b3f84db0ddf9f455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:43.212764 4383 raft_consensus.cc:2468] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:43.214394 4653 tablet_bootstrap.cc:492] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:43.214871 4653 ts_tablet_manager.cc:1403] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.057s user 0.010s sys 0.004s
W20260430 19:58:43.216115 4023 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d1d52994ff694886b7d4ff4224367082 consensus state has no leader
I20260430 19:58:43.216945 4023 master_service.cc:954] ReplaceTablet: received request to replace tablet 8a24d4962ee542e2a66c6e48b9a2b692 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:43.217386 4653 raft_consensus.cc:359] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.218036 4653 raft_consensus.cc:385] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.218302 4653 raft_consensus.cc:740] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.221103 4653 consensus_queue.cc:260] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.223187 4653 ts_tablet_manager.cc:1434] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260430 19:58:43.224426 4653 tablet_bootstrap.cc:492] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:43.227613 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 8e242c6ac7ea44a3b4f99b5d15f35c61
I20260430 19:58:43.229343 4022 catalog_manager.cc:6741] ReplaceTablet: tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 deleted and replaced by tablet e42a2e81e4a44cc29f352d8d55a03c4a
I20260430 19:58:43.231201 4204 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 with delete_type TABLET_DATA_DELETED (replaced by tablet e42a2e81e4a44cc29f352d8d55a03c4a) from {username='slave'} at 127.0.0.1:41996
W20260430 19:58:43.232681 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 already in progress: creating tablet
W20260430 19:58:43.233160 4025 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a7a82e60c974997b52a26650b131fc0 consensus state has no leader
I20260430 19:58:43.238157 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 with delete_type TABLET_DATA_DELETED (replaced by tablet e42a2e81e4a44cc29f352d8d55a03c4a) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:43.238157 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d with delete_type TABLET_DATA_DELETED (replaced by tablet d1d52994ff694886b7d4ff4224367082) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:43.239313 4657 ts_tablet_manager.cc:1929] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.239687 4657 log.cc:1199] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:43.240312 4657 ts_tablet_manager.cc:1950] T 8a24d4962ee542e2a66c6e48b9a2b692 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:43.242650 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 8a24d4962ee542e2a66c6e48b9a2b692 (table default.Table_4a891ffb2bad4492b969437e45c01394 [id=9dd258a7ab884edca842031038fa5c5c]) successfully deleted
I20260430 19:58:43.242854 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 with delete_type TABLET_DATA_DELETED (replaced by tablet e42a2e81e4a44cc29f352d8d55a03c4a) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:43.245249 4658 tablet_bootstrap.cc:492] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:43.245656 4658 ts_tablet_manager.cc:1403] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.123s user 0.012s sys 0.011s
W20260430 19:58:43.245939 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 already in progress: creating tablet
I20260430 19:58:43.249583 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e" } modify_external_catalogs: true
I20260430 19:58:43.250195 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e" } modify_external_catalogs: true
W20260430 19:58:43.251825 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6856b41ce4ae450ca7b08015666c3a2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6856b41ce4ae450ca7b08015666c3a2d already in progress: creating tablet
I20260430 19:58:43.253675 4652 ts_tablet_manager.cc:1434] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.158s user 0.005s sys 0.017s
I20260430 19:58:43.254648 4652 tablet_bootstrap.cc:492] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:43.260550 4024 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_d5d25ec643f44b2190b0473839ea4da1 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 3 got 0
I20260430 19:58:43.270833 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ce9a7d51fd524fa6a8930979d0c5b125"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:43.273334 4653 tablet_bootstrap.cc:654] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.257280 4658 raft_consensus.cc:359] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:43.275494 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce9a7d51fd524fa6a8930979d0c5b125 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.273631 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d with delete_type TABLET_DATA_DELETED (replaced by tablet d1d52994ff694886b7d4ff4224367082) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:43.277184 4655 tablet_replica.cc:333] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:43.277873 4655 raft_consensus.cc:2243] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:43.278301 4655 raft_consensus.cc:2272] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.278614 4658 raft_consensus.cc:385] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.278934 4658 raft_consensus.cc:740] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.279541 4658 consensus_queue.cc:260] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.281118 4655 ts_tablet_manager.cc:1916] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:43.281204 4652 tablet_bootstrap.cc:654] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.283411 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:42 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:43.288794 4658 ts_tablet_manager.cc:1434] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.043s user 0.005s sys 0.003s
I20260430 19:58:43.294080 4658 tablet_bootstrap.cc:492] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:43.296685 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a24d4962ee542e2a66c6e48b9a2b692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:42 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:43.276670 4027 catalog_manager.cc:5671] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:43.297843 4027 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_d5d25ec643f44b2190b0473839ea4da1 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 3 got 0
I20260430 19:58:43.302516 4652 tablet_bootstrap.cc:492] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:43.303074 4652 ts_tablet_manager.cc:1403] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.009s
I20260430 19:58:43.306018 4652 raft_consensus.cc:359] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.308199 4652 raft_consensus.cc:385] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.308485 4652 raft_consensus.cc:740] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
W20260430 19:58:43.309240 4686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:43.309345 4652 consensus_queue.cc:260] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.311273 4652 ts_tablet_manager.cc:1434] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260430 19:58:43.312381 4652 tablet_bootstrap.cc:492] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:43.318879 4652 tablet_bootstrap.cc:654] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.333621 4654 ts_tablet_manager.cc:1929] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.336524 4654 log.cc:1199] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/6856b41ce4ae450ca7b08015666c3a2d
I20260430 19:58:43.337323 4654 ts_tablet_manager.cc:1950] T 6856b41ce4ae450ca7b08015666c3a2d P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:43.338388 4652 tablet_bootstrap.cc:492] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:43.338858 4652 ts_tablet_manager.cc:1403] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.027s user 0.013s sys 0.011s
W20260430 19:58:43.341619 4015 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_d5d25ec643f44b2190b0473839ea4da1 [id=2302a4f9fee54fa6acbd21e0f5bbd31f], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6a7a82e60c974997b52a26650b131fc0 consensus state has no leader
I20260430 19:58:43.341326 4652 raft_consensus.cc:359] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.341929 4652 raft_consensus.cc:385] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.342202 4652 raft_consensus.cc:740] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.343992 4652 consensus_queue.cc:260] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.345669 4658 tablet_bootstrap.cc:654] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.348531 4652 ts_tablet_manager.cc:1434] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260430 19:58:43.349514 4652 tablet_bootstrap.cc:492] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:43.358484 4652 tablet_bootstrap.cc:654] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.358551 4658 tablet_bootstrap.cc:492] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:43.358973 4658 ts_tablet_manager.cc:1403] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.012s
I20260430 19:58:43.362102 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 6856b41ce4ae450ca7b08015666c3a2d (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:43.362074 4658 raft_consensus.cc:359] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.362780 4658 raft_consensus.cc:385] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.363080 4658 raft_consensus.cc:740] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.363876 4658 consensus_queue.cc:260] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.365091 4654 tablet_replica.cc:333] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:43.366108 4658 ts_tablet_manager.cc:1434] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260430 19:58:43.367441 4658 tablet_bootstrap.cc:492] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:43.377298 4652 tablet_bootstrap.cc:492] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:43.377152 4028 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet d0f228d11b9641589825bb6a61310672 (table default.Table_2cc5216057ba4cd688be5c6fbd4a6f69 [id=50df359f50de4b88a31deb7922615f33]). Expected version 1 got 0
I20260430 19:58:43.377743 4652 ts_tablet_manager.cc:1403] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.008s
I20260430 19:58:43.380268 4652 raft_consensus.cc:359] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.380968 4652 raft_consensus.cc:385] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.381217 4652 raft_consensus.cc:740] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.381990 4652 consensus_queue.cc:260] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.384137 4652 ts_tablet_manager.cc:1434] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260430 19:58:43.385021 4652 tablet_bootstrap.cc:492] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:43.391592 4652 tablet_bootstrap.cc:654] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.395006 4021 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 6108085995074ea890cc6933bf45661f
I20260430 19:58:43.397657 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 6108085995074ea890cc6933bf45661f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:43.399462 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6108085995074ea890cc6933bf45661f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6108085995074ea890cc6933bf45661f already in progress: creating tablet
I20260430 19:58:43.403901 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 6108085995074ea890cc6933bf45661f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:43.410176 4653 tablet_bootstrap.cc:492] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:43.410826 4653 ts_tablet_manager.cc:1403] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.187s user 0.008s sys 0.005s
I20260430 19:58:43.374792 4658 tablet_bootstrap.cc:654] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.397039 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 6108085995074ea890cc6933bf45661f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:43.416574 4655 ts_tablet_manager.cc:1929] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.418054 4669 raft_consensus.cc:493] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:43.418812 4669 raft_consensus.cc:515] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.374709 4654 raft_consensus.cc:2243] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:43.423310 4654 raft_consensus.cc:2272] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.431363 4669 leader_election.cc:290] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:43.417508 4653 raft_consensus.cc:359] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.434811 4653 raft_consensus.cc:385] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.435675 4653 raft_consensus.cc:740] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.437145 4653 consensus_queue.cc:260] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.440155 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a24d4962ee542e2a66c6e48b9a2b692" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:43.440728 4383 raft_consensus.cc:2468] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:43.444000 4021 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_2cc5216057ba4cd688be5c6fbd4a6f69" } new_table_name: "default.Table_5ea41a3ae89e4e45b01d9a69e437c571" modify_external_catalogs: true
I20260430 19:58:43.447808 4029 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_12e78c5061bc4945b51efe881ca1d3f1" } new_table_name: "default.Table_0dcf8ddbe6eb490b819da1d1cbc96508" modify_external_catalogs: true
I20260430 19:58:43.454044 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 504523705ff944c7b3f84db0ddf9f455
I20260430 19:58:43.457512 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 504523705ff944c7b3f84db0ddf9f455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:43.459717 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 504523705ff944c7b3f84db0ddf9f455 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 504523705ff944c7b3f84db0ddf9f455 already in progress: creating tablet
I20260430 19:58:43.460541 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 504523705ff944c7b3f84db0ddf9f455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:43.436429 4655 log.cc:1199] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/6856b41ce4ae450ca7b08015666c3a2d
I20260430 19:58:43.442258 4161 leader_election.cc:304] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:43.471640 4655 ts_tablet_manager.cc:1950] T 6856b41ce4ae450ca7b08015666c3a2d P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:43.472545 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_8ca4ebcc79d84b509500c61177e7d816"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:43.473444 4654 ts_tablet_manager.cc:1916] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:43.474283 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ca4ebcc79d84b509500c61177e7d816 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.476188 4669 raft_consensus.cc:2804] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:43.476534 4669 raft_consensus.cc:493] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:43.476846 4669 raft_consensus.cc:3060] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:43.488067 4653 ts_tablet_manager.cc:1434] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.077s user 0.004s sys 0.002s
I20260430 19:58:43.494230 4655 tablet_replica.cc:333] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:43.498345 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 6856b41ce4ae450ca7b08015666c3a2d (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:43.499827 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 504523705ff944c7b3f84db0ddf9f455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:43.500900 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 with delete_type TABLET_DATA_DELETED (replaced by tablet e42a2e81e4a44cc29f352d8d55a03c4a) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:43.508919 4028 master_service.cc:954] ReplaceTablet: received request to replace tablet d1d52994ff694886b7d4ff4224367082 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:43.512526 4024 master_service.cc:954] ReplaceTablet: received request to replace tablet 6108085995074ea890cc6933bf45661f from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:43.513384 4669 raft_consensus.cc:515] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.516258 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a24d4962ee542e2a66c6e48b9a2b692" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:43.516784 4383 raft_consensus.cc:3060] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:43.518306 4158 leader_election.cc:343] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:43.523223 4383 raft_consensus.cc:2468] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:43.524202 4161 leader_election.cc:304] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:43.526902 4700 raft_consensus.cc:2804] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:43.530862 4658 tablet_bootstrap.cc:492] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:43.531860 4653 tablet_bootstrap.cc:492] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:43.532238 4658 ts_tablet_manager.cc:1403] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.165s user 0.015s sys 0.012s
I20260430 19:58:43.539608 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_189237ca9d4140a59b16e5afb2196d09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:43.541316 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_189237ca9d4140a59b16e5afb2196d09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.547284 4658 raft_consensus.cc:359] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.547937 4658 raft_consensus.cc:385] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.548183 4658 raft_consensus.cc:740] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.548789 4658 consensus_queue.cc:260] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.566740 4658 ts_tablet_manager.cc:1434] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260430 19:58:43.573803 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_d904dde10c354ae5ab6b325322b1c0ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:43.575532 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d904dde10c354ae5ab6b325322b1c0ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.580122 4658 tablet_bootstrap.cc:492] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:43.600255 4669 leader_election.cc:290] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
W20260430 19:58:43.603856 4158 leader_election.cc:343] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:43.615697 4658 tablet_bootstrap.cc:654] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.618683 4652 tablet_bootstrap.cc:492] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:43.619246 4652 ts_tablet_manager.cc:1403] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.234s user 0.064s sys 0.070s
I20260430 19:58:43.620322 4658 tablet_bootstrap.cc:492] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:43.620694 4658 ts_tablet_manager.cc:1403] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.048s user 0.005s sys 0.005s
I20260430 19:58:43.622943 4658 raft_consensus.cc:359] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.623610 4658 raft_consensus.cc:385] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.623874 4658 raft_consensus.cc:740] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.624457 4658 consensus_queue.cc:260] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.632290 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ec013029c0de4691aee33eec63af4b84" } modify_external_catalogs: true
I20260430 19:58:43.632836 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ec013029c0de4691aee33eec63af4b84" } modify_external_catalogs: true
I20260430 19:58:43.633507 4658 ts_tablet_manager.cc:1434] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20260430 19:58:43.639370 4652 raft_consensus.cc:359] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.640185 4700 raft_consensus.cc:697] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:43.641788 4700 consensus_queue.cc:237] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.643618 4653 tablet_bootstrap.cc:654] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.646482 4658 tablet_bootstrap.cc:492] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:43.647415 4652 raft_consensus.cc:385] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.648463 4652 raft_consensus.cc:740] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.649224 4652 consensus_queue.cc:260] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.657823 4225 raft_consensus.cc:1275] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Refusing update from remote peer 8a11a9a7f244435b8e84554e3fabbd0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:43.659389 4658 tablet_bootstrap.cc:654] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.660065 4662 consensus_queue.cc:1048] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:43.664652 4652 ts_tablet_manager.cc:1434] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.045s user 0.004s sys 0.001s
I20260430 19:58:43.675596 4652 tablet_bootstrap.cc:492] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:43.679061 4383 raft_consensus.cc:1275] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Refusing update from remote peer 8a11a9a7f244435b8e84554e3fabbd0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:43.680845 4655 raft_consensus.cc:2243] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:43.681515 4662 consensus_queue.cc:1048] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:43.687150 4655 pending_rounds.cc:70] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Trying to abort 1 pending ops.
I20260430 19:58:43.687788 4655 pending_rounds.cc:77] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964090461130752 op_type: NO_OP noop_request { }
I20260430 19:58:43.688346 4655 raft_consensus.cc:2889] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:43.690372 4652 tablet_bootstrap.cc:654] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.692059 4655 raft_consensus.cc:2272] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.694643 4655 ts_tablet_manager.cc:1916] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:43.713831 4029 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 238e99c55dd6480da589cf3f43483773 has no consensus state
I20260430 19:58:43.716238 4652 tablet_bootstrap.cc:492] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:43.716630 4652 ts_tablet_manager.cc:1403] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.041s user 0.009s sys 0.001s
I20260430 19:58:43.719013 4652 raft_consensus.cc:359] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.719622 4652 raft_consensus.cc:385] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.719867 4652 raft_consensus.cc:740] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.720535 4652 consensus_queue.cc:260] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.731689 4652 ts_tablet_manager.cc:1434] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260430 19:58:43.740048 4702 mvcc.cc:204] Tried to move back new op lower bound from 7280964090420305920 to 7280964088606785536. Current Snapshot: MvccSnapshot[applied={T|T < 7280964090420305920}]
I20260430 19:58:43.751076 4706 mvcc.cc:204] Tried to move back new op lower bound from 7280964090420305920 to 7280964088606785536. Current Snapshot: MvccSnapshot[applied={T|T < 7280964090420305920}]
I20260430 19:58:43.764348 4714 tablet.cc:1724] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260430 19:58:43.773631 4655 ts_tablet_manager.cc:1929] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:43.774075 4655 log.cc:1199] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:43.774901 4655 ts_tablet_manager.cc:1950] T 8a24d4962ee542e2a66c6e48b9a2b692 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:43.777555 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 8a24d4962ee542e2a66c6e48b9a2b692 (table default.Table_4a891ffb2bad4492b969437e45c01394 [id=9dd258a7ab884edca842031038fa5c5c]) successfully deleted
I20260430 19:58:43.797520 4707 mvcc.cc:204] Tried to move back new op lower bound from 7280964090420305920 to 7280964088606785536. Current Snapshot: MvccSnapshot[applied={T|T < 7280964090420305920}]
W20260430 19:58:43.803148 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 6108085995074ea890cc6933bf45661f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6108085995074ea890cc6933bf45661f already in progress: creating tablet
I20260430 19:58:43.804162 4711 raft_consensus.cc:493] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:43.804868 4711 raft_consensus.cc:515] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.809460 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1d52994ff694886b7d4ff4224367082" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:43.809844 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 504523705ff944c7b3f84db0ddf9f455 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 504523705ff944c7b3f84db0ddf9f455 already in progress: creating tablet
I20260430 19:58:43.810063 4384 raft_consensus.cc:2468] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:43.811228 4470 leader_election.cc:304] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:43.811669 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1d52994ff694886b7d4ff4224367082" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:43.812086 4681 raft_consensus.cc:2804] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:43.812382 4681 raft_consensus.cc:493] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:43.812664 4681 raft_consensus.cc:3060] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:43.813156 4470 leader_election.cc:343] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:43.813700 4655 tablet_replica.cc:333] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:43.816514 4655 raft_consensus.cc:2243] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:43.817107 4655 raft_consensus.cc:2272] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.823262 4681 raft_consensus.cc:515] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.839370 4655 ts_tablet_manager.cc:1916] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:43.850162 4712 tablet.cc:1724] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260430 19:58:43.855911 4653 tablet_bootstrap.cc:492] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:43.856343 4653 ts_tablet_manager.cc:1403] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.325s user 0.009s sys 0.015s
I20260430 19:58:43.858976 4654 ts_tablet_manager.cc:1929] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.859373 4654 log.cc:1199] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/8e242c6ac7ea44a3b4f99b5d15f35c61
I20260430 19:58:43.859894 4653 raft_consensus.cc:359] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.860641 4653 raft_consensus.cc:385] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.861060 4653 raft_consensus.cc:740] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.863469 4653 consensus_queue.cc:260] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.867486 4653 ts_tablet_manager.cc:1434] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.011s user 0.003s sys 0.004s
I20260430 19:58:43.868512 4653 tablet_bootstrap.cc:492] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:43.868659 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_d5d25ec643f44b2190b0473839ea4da1" } modify_external_catalogs: true
I20260430 19:58:43.869421 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_d5d25ec643f44b2190b0473839ea4da1" } modify_external_catalogs: true
I20260430 19:58:43.874894 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1d52994ff694886b7d4ff4224367082" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:43.880215 4654 ts_tablet_manager.cc:1950] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:43.880439 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1d52994ff694886b7d4ff4224367082" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:43.881035 4383 raft_consensus.cc:3060] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:43.876276 4470 leader_election.cc:343] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:43.871376 4711 leader_election.cc:290] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:43.878690 4717 tablet.cc:1724] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260430 19:58:43.873157 4681 leader_election.cc:290] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:43.865293 4658 tablet_bootstrap.cc:492] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:43.887745 4658 ts_tablet_manager.cc:1403] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.241s user 0.028s sys 0.035s
I20260430 19:58:43.889127 4653 tablet_bootstrap.cc:654] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:43.905505 4711 raft_consensus.cc:493] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:43.905956 4711 raft_consensus.cc:515] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.907725 4711 leader_election.cc:290] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:43.910152 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_46b80f7ea2fc4fb5aadb474e5471fbb3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:43.911849 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46b80f7ea2fc4fb5aadb474e5471fbb3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:43.917222 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "504523705ff944c7b3f84db0ddf9f455" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:43.922453 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "504523705ff944c7b3f84db0ddf9f455" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:43.923327 4653 tablet_bootstrap.cc:492] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
W20260430 19:58:43.924397 4470 leader_election.cc:343] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:43.929906 4714 tablet.cc:1719] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Skipping requested alter to schema version 3, tablet already version 3
I20260430 19:58:43.931318 4658 raft_consensus.cc:359] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.932590 4667 raft_consensus.cc:493] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:43.933043 4667 raft_consensus.cc:515] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.934151 4653 ts_tablet_manager.cc:1403] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.055s user 0.010s sys 0.002s
I20260430 19:58:43.935873 4667 leader_election.cc:290] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
W20260430 19:58:43.939558 4470 leader_election.cc:343] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:43.939970 4470 leader_election.cc:304] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:43.940675 4681 raft_consensus.cc:2749] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:43.932103 4658 raft_consensus.cc:385] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.942659 4658 raft_consensus.cc:740] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.943926 4658 consensus_queue.cc:260] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.945220 4712 tablet.cc:1719] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Skipping requested alter to schema version 3, tablet already version 3
I20260430 19:58:43.949744 4658 ts_tablet_manager.cc:1434] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
I20260430 19:58:43.950534 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0f228d11b9641589825bb6a61310672" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:43.951310 4534 raft_consensus.cc:2468] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:43.951541 4653 raft_consensus.cc:359] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.953100 4653 raft_consensus.cc:385] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:43.953393 4653 raft_consensus.cc:740] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:43.953857 4317 leader_election.cc:304] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:43.954788 4667 raft_consensus.cc:2804] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:43.955132 4667 raft_consensus.cc:493] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:43.955448 4667 raft_consensus.cc:3060] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:43.957803 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 (table default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]) successfully deleted
I20260430 19:58:43.958361 4654 tablet_replica.cc:333] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:43.959429 4655 ts_tablet_manager.cc:1929] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.959813 4655 log.cc:1199] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/8e242c6ac7ea44a3b4f99b5d15f35c61
I20260430 19:58:43.954012 4653 consensus_queue.cc:260] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:43.960783 4654 raft_consensus.cc:2243] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:43.961980 4654 raft_consensus.cc:2272] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.963804 4654 ts_tablet_manager.cc:1916] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:43.966831 4653 ts_tablet_manager.cc:1434] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.032s user 0.002s sys 0.004s
I20260430 19:58:43.967725 4653 tablet_bootstrap.cc:492] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:43.970023 4655 ts_tablet_manager.cc:1950] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:43.977393 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 (table default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]) successfully deleted
I20260430 19:58:43.986649 4667 raft_consensus.cc:515] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:43.993602 4654 ts_tablet_manager.cc:1929] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:43.993978 4654 log.cc:1199] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/8a24d4962ee542e2a66c6e48b9a2b692
I20260430 19:58:43.994654 4654 ts_tablet_manager.cc:1950] T 8a24d4962ee542e2a66c6e48b9a2b692 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:43.996541 4029 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_d5d25ec643f44b2190b0473839ea4da1 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]). Expected version 3 got 0
I20260430 19:58:43.997022 4654 tablet_replica.cc:333] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:43.997745 4654 raft_consensus.cc:2243] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:43.998186 4654 raft_consensus.cc:2272] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:43.998569 4020 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet d0f228d11b9641589825bb6a61310672 (table default.Table_2cc5216057ba4cd688be5c6fbd4a6f69 [id=50df359f50de4b88a31deb7922615f33]). Expected version 1 got 0
I20260430 19:58:43.998965 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0f228d11b9641589825bb6a61310672" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:43.999382 4020 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet d1d52994ff694886b7d4ff4224367082 (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:43.999480 4534 raft_consensus.cc:3060] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:44.005903 4534 raft_consensus.cc:2468] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:44.006923 4317 leader_election.cc:304] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
W20260430 19:58:44.009550 4021 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d0f228d11b9641589825bb6a61310672 consensus state has no leader
I20260430 19:58:44.010443 4654 ts_tablet_manager.cc:1916] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:44.010623 4653 tablet_bootstrap.cc:654] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.012344 4513 tablet_service.cc:1511] Processing CreateTablet for tablet a8b08893b8a5417392dcb7bc957c938b (DEFAULT_TABLE table=default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.013430 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8b08893b8a5417392dcb7bc957c938b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.015381 4514 tablet_service.cc:1511] Processing CreateTablet for tablet d7aa58a597da45ca8c00d26a615e10bb (DEFAULT_TABLE table=default.Table_e4bcca1db8ad434780968ac6fe236a2c [id=06859fba4a5d4b8f978e6f56edc5fb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.014146 4667 leader_election.cc:290] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:44.016444 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aa58a597da45ca8c00d26a615e10bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.018652 4667 raft_consensus.cc:2804] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:44.022872 4202 tablet_service.cc:1511] Processing CreateTablet for tablet a8b08893b8a5417392dcb7bc957c938b (DEFAULT_TABLE table=default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.024014 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8b08893b8a5417392dcb7bc957c938b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.024048 4667 raft_consensus.cc:697] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:44.027601 4205 tablet_service.cc:1511] Processing CreateTablet for tablet d7aa58a597da45ca8c00d26a615e10bb (DEFAULT_TABLE table=default.Table_e4bcca1db8ad434780968ac6fe236a2c [id=06859fba4a5d4b8f978e6f56edc5fb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.028805 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aa58a597da45ca8c00d26a615e10bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.032145 4653 tablet_bootstrap.cc:492] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:44.032562 4653 ts_tablet_manager.cc:1403] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.065s user 0.009s sys 0.001s
I20260430 19:58:44.031144 4667 consensus_queue.cc:237] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.040982 4363 tablet_service.cc:1511] Processing CreateTablet for tablet a8b08893b8a5417392dcb7bc957c938b (DEFAULT_TABLE table=default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.042140 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8b08893b8a5417392dcb7bc957c938b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.045171 4653 raft_consensus.cc:359] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.045799 4653 raft_consensus.cc:385] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.046047 4653 raft_consensus.cc:740] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.039178 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 8a24d4962ee542e2a66c6e48b9a2b692 (table default.Table_4a891ffb2bad4492b969437e45c01394 [id=9dd258a7ab884edca842031038fa5c5c]) successfully deleted
I20260430 19:58:44.046650 4653 consensus_queue.cc:260] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.075765 4512 tablet_service.cc:1511] Processing CreateTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a (DEFAULT_TABLE table=default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.077384 4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42a2e81e4a44cc29f352d8d55a03c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.083977 4364 tablet_service.cc:1511] Processing CreateTablet for tablet d7aa58a597da45ca8c00d26a615e10bb (DEFAULT_TABLE table=default.Table_e4bcca1db8ad434780968ac6fe236a2c [id=06859fba4a5d4b8f978e6f56edc5fb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.085052 4653 ts_tablet_manager.cc:1434] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.052s user 0.007s sys 0.000s
I20260430 19:58:44.085196 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aa58a597da45ca8c00d26a615e10bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.099282 4653 tablet_bootstrap.cc:492] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:44.101362 4204 tablet_service.cc:1511] Processing CreateTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a (DEFAULT_TABLE table=default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.103039 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42a2e81e4a44cc29f352d8d55a03c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.113876 4681 raft_consensus.cc:493] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.114380 4681 raft_consensus.cc:515] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.116122 4681 leader_election.cc:290] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
W20260430 19:58:44.120072 4470 leader_election.cc:343] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: 6856b41ce4ae450ca7b08015666c3a2d
W20260430 19:58:44.120687 4470 leader_election.cc:343] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Not found: Tablet not found: 6856b41ce4ae450ca7b08015666c3a2d
I20260430 19:58:44.121037 4470 leader_election.cc:304] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:44.122073 4681 raft_consensus.cc:2749] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:44.129460 4361 tablet_service.cc:1511] Processing CreateTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a (DEFAULT_TABLE table=default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:44.130928 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42a2e81e4a44cc29f352d8d55a03c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:44.133136 4653 tablet_bootstrap.cc:654] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.157061 4383 raft_consensus.cc:2468] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:44.158353 4470 leader_election.cc:304] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:44.159560 4681 raft_consensus.cc:2804] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:44.160018 4681 raft_consensus.cc:697] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:44.160786 4681 consensus_queue.cc:237] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.169555 4653 tablet_bootstrap.cc:492] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:44.170024 4653 ts_tablet_manager.cc:1403] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.071s user 0.005s sys 0.008s
I20260430 19:58:44.176862 4024 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_8ca4ebcc79d84b509500c61177e7d816" } new_table_name: "default.Table_4ffb858382fa495f8af004403e5ea2db" modify_external_catalogs: true
I20260430 19:58:44.181494 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_97bc319789364b85a7fba98b96b46157"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:44.183389 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97bc319789364b85a7fba98b96b46157 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:44.185935 4653 raft_consensus.cc:359] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.186605 4653 raft_consensus.cc:385] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.186877 4653 raft_consensus.cc:740] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.187489 4653 consensus_queue.cc:260] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:44.201084 4015 consensus_peers.cc:597] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 -> Peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Couldn't send request to peer 6b0a440dabed4814aaefe431944a7b5e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260430 19:58:44.215103 4653 ts_tablet_manager.cc:1434] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.045s user 0.005s sys 0.002s
I20260430 19:58:44.262162 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_e0abcc152ab14699b99ab62f4bf1b0f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:44.262890 4681 raft_consensus.cc:493] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.263299 4681 raft_consensus.cc:515] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:44.263922 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0abcc152ab14699b99ab62f4bf1b0f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:44.265043 4681 leader_election.cc:290] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
W20260430 19:58:44.266777 4317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 259908us
W20260430 19:58:44.268307 4470 leader_election.cc:343] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: 8e242c6ac7ea44a3b4f99b5d15f35c61
W20260430 19:58:44.274966 4470 leader_election.cc:343] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Not found: Tablet not found: 8e242c6ac7ea44a3b4f99b5d15f35c61
I20260430 19:58:44.275359 4470 leader_election.cc:304] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:44.276345 4681 raft_consensus.cc:2749] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:44.284358 4023 master_service.cc:954] ReplaceTablet: received request to replace tablet 6a7a82e60c974997b52a26650b131fc0 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:44.285938 4681 raft_consensus.cc:493] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.286370 4681 raft_consensus.cc:515] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.288185 4681 leader_election.cc:290] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:44.289765 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6108085995074ea890cc6933bf45661f" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:44.290346 4224 raft_consensus.cc:2468] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:44.291446 4470 leader_election.cc:304] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:44.292270 4681 raft_consensus.cc:2804] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:44.292587 4681 raft_consensus.cc:493] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:44.293479 4681 raft_consensus.cc:3060] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:44.297183 4700 raft_consensus.cc:493] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.296087 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6108085995074ea890cc6933bf45661f" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:44.297616 4700 raft_consensus.cc:515] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:44.299130 4470 leader_election.cc:343] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:44.299477 4700 leader_election.cc:290] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:44.300844 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0f228d11b9641589825bb6a61310672" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:44.301578 4534 raft_consensus.cc:2393] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a7356e99e744525861e12c108cfa3fe in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:44.303555 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0f228d11b9641589825bb6a61310672" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:44.305320 4161 leader_election.cc:304] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:44.306501 4700 raft_consensus.cc:3060] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:44.313597 4700 raft_consensus.cc:2749] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:44.315641 4681 raft_consensus.cc:515] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.322608 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6108085995074ea890cc6933bf45661f" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
W20260430 19:58:44.323707 4470 leader_election.cc:343] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:44.326728 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6108085995074ea890cc6933bf45661f" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:44.326277 4681 leader_election.cc:290] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:44.327251 4224 raft_consensus.cc:3060] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:44.334036 4224 raft_consensus.cc:2468] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:44.335218 4470 leader_election.cc:304] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:44.336385 4681 raft_consensus.cc:2804] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:44.340281 4700 raft_consensus.cc:493] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.340785 4700 raft_consensus.cc:515] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.340945 4681 raft_consensus.cc:697] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:44.343529 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "504523705ff944c7b3f84db0ddf9f455" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:44.344094 4383 raft_consensus.cc:2468] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:44.342303 4681 consensus_queue.cc:237] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.344894 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "504523705ff944c7b3f84db0ddf9f455" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:44.345448 4534 raft_consensus.cc:2468] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:44.347362 4158 leader_election.cc:304] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
W20260430 19:58:44.350450 4014 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 238e99c55dd6480da589cf3f43483773 (table default.Table_0dcf8ddbe6eb490b819da1d1cbc96508 [id=b43a4d5da8c84baa89c5eefa1fbd8409], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 238e99c55dd6480da589cf3f43483773 has no consensus state
I20260430 19:58:44.363097 4729 tablet_bootstrap.cc:492] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:44.381130 4729 tablet_bootstrap.cc:654] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.384771 4717 tablet.cc:1719] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Skipping requested alter to schema version 3, tablet already version 3
I20260430 19:58:44.405239 4700 leader_election.cc:290] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:44.405766 4700 raft_consensus.cc:2804] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:44.406109 4700 raft_consensus.cc:493] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:44.406426 4700 raft_consensus.cc:3060] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:44.410250 4158 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 62944.8us
I20260430 19:58:44.411404 4658 tablet_bootstrap.cc:492] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:44.417622 4658 tablet_bootstrap.cc:654] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.431113 4729 tablet_bootstrap.cc:492] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:44.431979 4729 ts_tablet_manager.cc:1403] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.071s user 0.016s sys 0.007s
I20260430 19:58:44.432860 4653 tablet_bootstrap.cc:492] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:44.434830 4729 raft_consensus.cc:359] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.435583 4729 raft_consensus.cc:385] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.435904 4729 raft_consensus.cc:740] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.436653 4729 consensus_queue.cc:260] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.438210 4653 tablet_bootstrap.cc:654] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.446899 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0f228d11b9641589825bb6a61310672" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:44.454712 4653 tablet_bootstrap.cc:492] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:44.455040 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet e42a2e81e4a44cc29f352d8d55a03c4a
I20260430 19:58:44.455202 4653 ts_tablet_manager.cc:1403] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.000s
I20260430 19:58:44.455564 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0f228d11b9641589825bb6a61310672" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:44.460839 4225 raft_consensus.cc:2468] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
W20260430 19:58:44.460963 4020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.3:35079 (request call id 11) took 899 ms (client timeout 1000 ms). Trace:
W20260430 19:58:44.461314 4020 rpcz_store.cc:269] 0430 19:58:43.561876 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:43.562670 (+ 794us) service_pool.cc:225] Handling call
0430 19:58:43.570923 (+ 8253us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:44.460867 (+889944us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":15918848}
I20260430 19:58:44.470849 4729 ts_tablet_manager.cc:1434] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.038s user 0.006s sys 0.000s
I20260430 19:58:44.478258 4729 tablet_bootstrap.cc:492] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:44.461707 4653 raft_consensus.cc:359] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.481819 4653 raft_consensus.cc:385] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.483983 4653 raft_consensus.cc:740] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.487280 4653 consensus_queue.cc:260] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.472407 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 with delete_type TABLET_DATA_DELETED (replaced by tablet e42a2e81e4a44cc29f352d8d55a03c4a) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:44.492336 4658 tablet_bootstrap.cc:492] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:44.492749 4658 ts_tablet_manager.cc:1403] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.008s
I20260430 19:58:44.493287 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:44.493692 4730 raft_consensus.cc:493] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.494190 4730 raft_consensus.cc:515] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.495249 4658 raft_consensus.cc:359] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.495859 4658 raft_consensus.cc:385] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.496091 4658 raft_consensus.cc:740] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.496980 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_189237ca9d4140a59b16e5afb2196d09" } new_table_name: "default.Table_a2efa2a9f9bd42738350b731f2debc69" modify_external_catalogs: true
I20260430 19:58:44.501569 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_b85a96ef82324def96137ff90837adf1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:44.503381 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b85a96ef82324def96137ff90837adf1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:44.509209 4658 consensus_queue.cc:260] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.511552 4658 ts_tablet_manager.cc:1434] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260430 19:58:44.508754 4653 ts_tablet_manager.cc:1434] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20260430 19:58:44.520681 4653 tablet_bootstrap.cc:492] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:44.521384 4658 tablet_bootstrap.cc:492] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:44.525748 4653 tablet_bootstrap.cc:654] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.492652 4364 tablet_service.cc:1558] Processing DeleteTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:44.496555 4729 tablet_bootstrap.cc:654] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.539397 4658 tablet_bootstrap.cc:654] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.519546 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:44.519042 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 6856b41ce4ae450ca7b08015666c3a2d with delete_type TABLET_DATA_DELETED (replaced by tablet d1d52994ff694886b7d4ff4224367082) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:44.546571 4654 ts_tablet_manager.cc:1929] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:44.547390 4654 log.cc:1199] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/6108085995074ea890cc6933bf45661f
I20260430 19:58:44.548858 4654 ts_tablet_manager.cc:1950] T 6108085995074ea890cc6933bf45661f P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:44.552398 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 16) took 1155 ms. Trace:
I20260430 19:58:44.554086 4737 tablet_replica.cc:333] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:44.554906 4737 raft_consensus.cc:2243] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:44.555459 4737 raft_consensus.cc:2272] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:44.557449 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8b08893b8a5417392dcb7bc957c938b" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:44.558949 4158 leader_election.cc:343] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:44.567345 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet e42a2e81e4a44cc29f352d8d55a03c4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e42a2e81e4a44cc29f352d8d55a03c4a already in progress: creating tablet
I20260430 19:58:44.569371 4730 leader_election.cc:290] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:44.571679 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 6a7a82e60c974997b52a26650b131fc0
I20260430 19:58:44.568380 4653 tablet_bootstrap.cc:492] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:44.574895 4653 ts_tablet_manager.cc:1403] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.054s user 0.010s sys 0.001s
I20260430 19:58:44.578326 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a7a82e60c974997b52a26650b131fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:44.580083 4029 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.1:59387 (request call id 23) took 847 ms (client timeout 1000 ms). Trace:
W20260430 19:58:44.580406 4029 rpcz_store.cc:269] 0430 19:58:43.732817 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:43.733046 (+ 229us) service_pool.cc:225] Handling call
0430 19:58:44.580025 (+846979us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260430 19:58:44.577104 4653 raft_consensus.cc:359] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.582321 4653 raft_consensus.cc:385] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.582556 4653 raft_consensus.cc:740] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.583243 4653 consensus_queue.cc:260] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.593770 4653 ts_tablet_manager.cc:1434] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260430 19:58:44.596522 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a7a82e60c974997b52a26650b131fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:44.597321 4742 tablet_replica.cc:333] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:44.598143 4742 raft_consensus.cc:2243] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:44.598742 4742 raft_consensus.cc:2272] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:44.601603 4742 ts_tablet_manager.cc:1916] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:44.603598 4653 tablet_bootstrap.cc:492] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:44.618428 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_0f760a2ab2ce43659a9fcbc0ea85758e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:44.620163 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f760a2ab2ce43659a9fcbc0ea85758e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:44.622845 4653 tablet_bootstrap.cc:654] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.632995 4700 raft_consensus.cc:515] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:44.644552 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet d1d52994ff694886b7d4ff4224367082
I20260430 19:58:44.645879 4028 catalog_manager.cc:6741] ReplaceTablet: tablet d1d52994ff694886b7d4ff4224367082 deleted and replaced by tablet c78d50690da7428fadb848dadefec47c
I20260430 19:58:44.646203 4028 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:55130 (request call id 46) took 1137 ms. Trace:
W20260430 19:58:44.648547 4025 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.2:49905 (request call id 19) took 1087 ms (client timeout 1000 ms). Trace:
W20260430 19:58:44.648887 4025 rpcz_store.cc:269] 0430 19:58:43.561537 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:43.562333 (+ 796us) service_pool.cc:225] Handling call
0430 19:58:43.577541 (+ 15208us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:44.648483 (+1070942us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":20031872}
I20260430 19:58:44.646503 4028 rpcz_store.cc:276] 0430 19:58:43.508540 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:43.508755 (+ 215us) service_pool.cc:225] Handling call
0430 19:58:43.562797 (+ 54042us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f292911a491 00007f2926d45227 00007f2926d451b2 00007f2926d450e1 00007f2926d4505c 00007f2926d44312 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:44.646149 (+1083352us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:44.625617 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:44.625987 (+ 370us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:44.626008 (+ 21us) write_op.cc:435] Acquired schema lock
0430 19:58:44.626021 (+ 13us) tablet.cc:662] Decoding operations
0430 19:58:44.626293 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:44.626333 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:44.626355 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0430 19:58:44.626540 (+ 185us) tablet.cc:701] Row locks acquired
0430 19:58:44.626553 (+ 13us) write_op.cc:260] PREPARE: finished
0430 19:58:44.626673 (+ 120us) write_op.cc:270] Start()
0430 19:58:44.626746 (+ 73us) write_op.cc:276] Timestamp: P: 1777579124626615 usec, L: 0
0430 19:58:44.626761 (+ 15us) op_driver.cc:348] REPLICATION: starting
0430 19:58:44.627199 (+ 438us) log.cc:844] Serialized 621 byte log entry
0430 19:58:44.641863 (+ 14664us) op_driver.cc:464] REPLICATION: finished
0430 19:58:44.642206 (+ 343us) write_op.cc:301] APPLY: starting
0430 19:58:44.642256 (+ 50us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:44.642354 (+ 98us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:44.642439 (+ 85us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:44.642779 (+ 340us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:44.642978 (+ 199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:44.643018 (+ 40us) write_op.cc:312] APPLY: finished
0430 19:58:44.643180 (+ 162us) log.cc:844] Serialized 28 byte log entry
0430 19:58:44.643311 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:44.643376 (+ 65us) write_op.cc:454] Released schema lock
0430 19:58:44.643404 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":60694400,"child_traces":[["op",{"apply.queue_time_us":201,"apply.run_cpu_time_us":1363,"apply.run_wall_time_us":1364,"num_ops":2,"prepare.queue_time_us":576204,"raft.queue_time_us":170,"raft.run_cpu_time_us":656,"raft.run_wall_time_us":655,"replication_time_us":14990}]]}
I20260430 19:58:44.672233 4729 tablet_bootstrap.cc:492] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:44.672778 4729 ts_tablet_manager.cc:1403] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.195s user 0.012s sys 0.008s
I20260430 19:58:44.677138 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet d1d52994ff694886b7d4ff4224367082 with delete_type TABLET_DATA_DELETED (replaced by tablet c78d50690da7428fadb848dadefec47c) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:44.687482 4025 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5ea41a3ae89e4e45b01d9a69e437c571" } modify_external_catalogs: true
I20260430 19:58:44.688328 4025 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5ea41a3ae89e4e45b01d9a69e437c571" } modify_external_catalogs: true
W20260430 19:58:44.703701 4024 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43d96039a1d64306b04ee49a6b86e854 has no consensus state
I20260430 19:58:44.711738 4737 ts_tablet_manager.cc:1916] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:44.560256 4658 tablet_bootstrap.cc:492] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:44.712478 4658 ts_tablet_manager.cc:1403] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.191s user 0.006s sys 0.004s
I20260430 19:58:44.578411 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8b08893b8a5417392dcb7bc957c938b" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:44.609009 4364 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a7a82e60c974997b52a26650b131fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:44.660531 4724 raft_consensus.cc:493] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.715010 4724 raft_consensus.cc:515] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.716442 4658 raft_consensus.cc:359] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.716960 4724 leader_election.cc:290] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:44.717101 4658 raft_consensus.cc:385] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.717382 4658 raft_consensus.cc:740] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.718082 4658 consensus_queue.cc:260] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.718690 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:44.719275 4534 raft_consensus.cc:2468] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:44.719800 4658 ts_tablet_manager.cc:1434] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260430 19:58:44.721208 4317 leader_election.cc:304] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:44.721841 4658 tablet_bootstrap.cc:492] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:44.722653 4724 raft_consensus.cc:2804] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:44.722977 4724 raft_consensus.cc:493] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:44.723266 4724 raft_consensus.cc:3060] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:44.663182 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet d1d52994ff694886b7d4ff4224367082 with delete_type TABLET_DATA_DELETED (replaced by tablet c78d50690da7428fadb848dadefec47c) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:44.733223 4724 raft_consensus.cc:515] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.737759 4729 raft_consensus.cc:359] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.747888 4729 raft_consensus.cc:385] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.745390 4658 tablet_bootstrap.cc:654] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:44.712368 4654 rpcz_store.cc:276] 0430 19:58:43.396696 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:43.396866 (+ 170us) service_pool.cc:225] Handling call
0430 19:58:43.996873 (+600007us) ts_tablet_manager.cc:1151] Deleting tablet 6108085995074ea890cc6933bf45661f
0430 19:58:43.996913 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:44.023191 (+ 26278us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:44.546434 (+523243us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:44.552331 (+ 5897us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":582535}
I20260430 19:58:44.759385 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 6108085995074ea890cc6933bf45661f (table default.Table_78209fb594f841308f82995e31eff165 [id=f509dee5b07f496381f8a0edcbb50ced]) successfully deleted
I20260430 19:58:44.760574 4654 tablet_replica.cc:333] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:44.765764 4654 raft_consensus.cc:2243] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:44.766337 4654 raft_consensus.cc:2272] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:44.768399 4654 ts_tablet_manager.cc:1916] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:44.773293 4658 tablet_bootstrap.cc:492] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:44.773763 4658 ts_tablet_manager.cc:1403] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.052s user 0.011s sys 0.001s
I20260430 19:58:44.778770 4658 raft_consensus.cc:359] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.779400 4658 raft_consensus.cc:385] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.779644 4658 raft_consensus.cc:740] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.780226 4658 consensus_queue.cc:260] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.802179 4729 raft_consensus.cc:740] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.804229 4658 ts_tablet_manager.cc:1434] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.030s user 0.008s sys 0.000s
I20260430 19:58:44.803961 4729 consensus_queue.cc:260] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.811964 4729 ts_tablet_manager.cc:1434] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.139s user 0.004s sys 0.002s
I20260430 19:58:44.819846 4724 leader_election.cc:290] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:44.642763 4653 tablet_bootstrap.cc:492] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:44.822152 4653 ts_tablet_manager.cc:1403] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.219s user 0.005s sys 0.006s
I20260430 19:58:44.662082 4730 raft_consensus.cc:493] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.676272 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet d1d52994ff694886b7d4ff4224367082 with delete_type TABLET_DATA_DELETED (replaced by tablet c78d50690da7428fadb848dadefec47c) from {username='slave'} at 127.0.0.1:41996
W20260430 19:58:44.715633 4161 leader_election.cc:343] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:44.720861 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:44.783764 4738 raft_consensus.cc:493] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.820952 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:44.824934 4653 raft_consensus.cc:359] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.828385 4653 raft_consensus.cc:385] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:44.828655 4653 raft_consensus.cc:740] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:44.827399 4738 raft_consensus.cc:515] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.828011 4730 raft_consensus.cc:515] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.833968 4730 leader_election.cc:290] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:44.830699 4161 leader_election.cc:304] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:44.835039 4744 consensus_queue.cc:1048] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260430 19:58:44.835544 4161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 119930us
I20260430 19:58:44.838776 4225 raft_consensus.cc:2468] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:44.844455 4738 leader_election.cc:290] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:44.844900 4738 raft_consensus.cc:2749] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:44.829209 4653 consensus_queue.cc:260] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.855572 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "504523705ff944c7b3f84db0ddf9f455" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:44.856515 4021 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_97bc319789364b85a7fba98b96b46157" } modify_external_catalogs: true
I20260430 19:58:44.857093 4021 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_97bc319789364b85a7fba98b96b46157" } modify_external_catalogs: true
I20260430 19:58:44.863447 4700 leader_election.cc:290] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:44.868675 4653 ts_tablet_manager.cc:1434] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
I20260430 19:58:44.872047 4654 ts_tablet_manager.cc:1929] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:44.872457 4654 log.cc:1199] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/504523705ff944c7b3f84db0ddf9f455
W20260430 19:58:44.876757 4317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 155603us
I20260430 19:58:44.878743 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:44.843145 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:44.879544 4383 raft_consensus.cc:2393] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a7356e99e744525861e12c108cfa3fe in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:44.879709 4534 raft_consensus.cc:2468] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:44.846871 4533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1d52994ff694886b7d4ff4224367082" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:44.855489 4470 consensus_peers.cc:597] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:44.863924 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "504523705ff944c7b3f84db0ddf9f455" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:44.880842 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1d52994ff694886b7d4ff4224367082" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:44.881320 4532 raft_consensus.cc:3060] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:44.881515 4383 raft_consensus.cc:2393] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a7356e99e744525861e12c108cfa3fe in current term 1: Already voted for candidate 8a11a9a7f244435b8e84554e3fabbd0f in this term.
W20260430 19:58:44.882321 4161 leader_election.cc:343] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:44.873099 4654 ts_tablet_manager.cc:1950] T 504523705ff944c7b3f84db0ddf9f455 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:44.887637 4742 ts_tablet_manager.cc:1929] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:44.888054 4742 log.cc:1199] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/6a7a82e60c974997b52a26650b131fc0
I20260430 19:58:44.890735 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 17) took 1431 ms. Trace:
I20260430 19:58:44.891443 4681 raft_consensus.cc:493] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.891105 4654 rpcz_store.cc:276] 0430 19:58:43.459330 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:43.460354 (+ 1024us) service_pool.cc:225] Handling call
0430 19:58:44.760317 (+1299963us) ts_tablet_manager.cc:1151] Deleting tablet 504523705ff944c7b3f84db0ddf9f455
0430 19:58:44.760454 (+ 137us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:44.830869 (+ 70415us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:44.871904 (+ 41035us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:44.890658 (+ 18754us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1299219}
I20260430 19:58:44.891885 4681 raft_consensus.cc:515] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.892596 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 504523705ff944c7b3f84db0ddf9f455 (table default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e [id=0f3d9fa7646043459be13fcce244dc0f]) successfully deleted
I20260430 19:58:44.894196 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:44.894928 4531 raft_consensus.cc:3060] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:44.898983 4158 leader_election.cc:304] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:44.894243 4681 leader_election.cc:290] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:44.900203 4158 leader_election.cc:304] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:44.903663 4532 raft_consensus.cc:2468] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:44.903751 4531 raft_consensus.cc:2468] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:44.905360 4317 leader_election.cc:304] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:44.905628 4749 consensus_queue.cc:1048] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:44.906754 4744 raft_consensus.cc:2804] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:44.907285 4744 raft_consensus.cc:697] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:44.908020 4744 consensus_queue.cc:237] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.910709 4742 ts_tablet_manager.cc:1950] T 6a7a82e60c974997b52a26650b131fc0 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:44.911931 4742 tablet_replica.cc:333] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:44.912652 4742 raft_consensus.cc:2243] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:44.913012 4730 raft_consensus.cc:2804] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:44.913323 4730 raft_consensus.cc:493] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:44.912875 4729 tablet_bootstrap.cc:492] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:44.913642 4730 raft_consensus.cc:3060] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:44.914243 4015 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.62:33781, user_credentials={real_user=slave}} blocked reactor thread for 109969us
I20260430 19:58:44.922844 4700 raft_consensus.cc:493] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:44.923295 4700 raft_consensus.cc:515] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:44.927934 4028 master_service.cc:954] ReplaceTablet: received request to replace tablet e42a2e81e4a44cc29f352d8d55a03c4a from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:44.950947 4654 tablet_replica.cc:333] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:44.924724 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8b08893b8a5417392dcb7bc957c938b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:44.957197 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ace9ebd59ec0464cbca1f928a48eedaa" } new_table_name: "default.Table_5e8fd88607e445828f7d27136d5116a9" modify_external_catalogs: true
W20260430 19:58:44.959187 4158 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 59051.7us
I20260430 19:58:44.960345 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_d5d25ec643f44b2190b0473839ea4da1 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]) successfully deleted
I20260430 19:58:44.965814 4750 raft_consensus.cc:2706] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Received pre-election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260430 19:58:44.974388 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8b08893b8a5417392dcb7bc957c938b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:44.975006 4383 raft_consensus.cc:2468] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
W20260430 19:58:44.982435 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 41df32f61a7f412eac7059f589388dae has no consensus state
I20260430 19:58:44.992080 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ccf1123d6d5d4e489838f28c6125aa15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:44.993803 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ccf1123d6d5d4e489838f28c6125aa15 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:44.996977 4654 raft_consensus.cc:2243] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:45.000103 4654 pending_rounds.cc:70] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:45.000465 4654 pending_rounds.cc:77] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964095771381760 op_type: NO_OP noop_request { }
I20260430 19:58:45.000857 4654 raft_consensus.cc:2889] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:45.001227 4654 raft_consensus.cc:2272] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.001037 4470 leader_election.cc:304] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:45.002422 4681 raft_consensus.cc:2804] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:45.002764 4681 raft_consensus.cc:493] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:45.003055 4681 raft_consensus.cc:3060] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.009315 4681 raft_consensus.cc:515] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.011780 4681 leader_election.cc:290] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
W20260430 19:58:45.014580 4753 consensus_queue.cc:1175] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260430 19:58:45.016093 4158 leader_election.cc:304] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:45.016889 4654 ts_tablet_manager.cc:1916] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.020385 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aa58a597da45ca8c00d26a615e10bb" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:45.020927 4383 raft_consensus.cc:2468] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:45.021986 4161 leader_election.cc:304] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:45.022567 4700 leader_election.cc:290] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:45.027838 4223 raft_consensus.cc:2468] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:45.018525 4730 raft_consensus.cc:515] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.032923 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8b08893b8a5417392dcb7bc957c938b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:45.033452 4383 raft_consensus.cc:3060] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.034091 4730 leader_election.cc:290] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:45.034749 4738 raft_consensus.cc:2804] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:45.035094 4738 raft_consensus.cc:493] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:45.035386 4738 raft_consensus.cc:3060] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.042425 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:45.044055 4729 tablet_bootstrap.cc:654] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:45.045469 4161 leader_election.cc:343] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:45.050407 4738 raft_consensus.cc:515] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.051937 4737 ts_tablet_manager.cc:1929] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.052357 4737 log.cc:1199] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/8e242c6ac7ea44a3b4f99b5d15f35c61
I20260430 19:58:45.052999 4737 ts_tablet_manager.cc:1950] T 8e242c6ac7ea44a3b4f99b5d15f35c61 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.054049 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aa58a597da45ca8c00d26a615e10bb" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:45.054582 4382 raft_consensus.cc:3060] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.058981 4383 raft_consensus.cc:2468] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:45.060006 4470 leader_election.cc:304] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
W20260430 19:58:45.060940 4470 consensus_peers.cc:597] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:45.061455 4681 raft_consensus.cc:2804] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:45.061625 4382 raft_consensus.cc:2468] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:45.061880 4681 raft_consensus.cc:697] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:45.062740 4161 leader_election.cc:304] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:45.063158 4681 consensus_queue.cc:237] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.065335 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 8e242c6ac7ea44a3b4f99b5d15f35c61 (table default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]) successfully deleted
I20260430 19:58:45.066452 4737 tablet_replica.cc:333] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:45.064126 4742 raft_consensus.cc:2272] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.070004 4737 raft_consensus.cc:2243] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.070458 4737 raft_consensus.cc:2272] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.070916 4750 raft_consensus.cc:2804] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:45.072234 4737 ts_tablet_manager.cc:1916] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.073530 4742 ts_tablet_manager.cc:1916] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.078168 4700 raft_consensus.cc:2804] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:45.078641 4700 raft_consensus.cc:697] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:45.079352 4700 consensus_queue.cc:237] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.083747 4738 leader_election.cc:290] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:45.086315 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8b08893b8a5417392dcb7bc957c938b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:45.086915 4223 raft_consensus.cc:3060] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:45.094873 4158 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 134723us
I20260430 19:58:45.097615 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42a2e81e4a44cc29f352d8d55a03c4a" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:45.102708 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aa58a597da45ca8c00d26a615e10bb" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
W20260430 19:58:45.105051 4158 leader_election.cc:343] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:45.105473 4158 leader_election.cc:304] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:45.106475 4760 raft_consensus.cc:2749] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260430 19:58:45.106703 4158 leader_election.cc:343] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:45.107641 4750 raft_consensus.cc:697] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:45.107584 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 43d96039a1d64306b04ee49a6b86e854 (DEFAULT_TABLE table=default.Table_4ffb858382fa495f8af004403e5ea2db [id=abddf7bc9a6c4a76aa323bf84a2cc21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.108820 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d96039a1d64306b04ee49a6b86e854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.108383 4750 consensus_queue.cc:237] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.117370 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 43d96039a1d64306b04ee49a6b86e854 (DEFAULT_TABLE table=default.Table_4ffb858382fa495f8af004403e5ea2db [id=abddf7bc9a6c4a76aa323bf84a2cc21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.118677 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d96039a1d64306b04ee49a6b86e854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.123494 4364 tablet_service.cc:1511] Processing CreateTablet for tablet 238e99c55dd6480da589cf3f43483773 (DEFAULT_TABLE table=default.Table_0dcf8ddbe6eb490b819da1d1cbc96508 [id=b43a4d5da8c84baa89c5eefa1fbd8409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.123822 4654 ts_tablet_manager.cc:1929] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:45.124158 4654 log.cc:1199] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/e42a2e81e4a44cc29f352d8d55a03c4a
I20260430 19:58:45.124675 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238e99c55dd6480da589cf3f43483773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.124784 4654 ts_tablet_manager.cc:1950] T e42a2e81e4a44cc29f352d8d55a03c4a P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:45.127630 4028 master_service.cc:954] ReplaceTablet: received request to replace tablet e42a2e81e4a44cc29f352d8d55a03c4a from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:45.128818 4023 catalog_manager.cc:5671] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2). New cstate: current_term: 1 leader_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:45.112618 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aa58a597da45ca8c00d26a615e10bb" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:45.139817 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 238e99c55dd6480da589cf3f43483773 (DEFAULT_TABLE table=default.Table_0dcf8ddbe6eb490b819da1d1cbc96508 [id=b43a4d5da8c84baa89c5eefa1fbd8409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.136253 4729 tablet_bootstrap.cc:492] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:45.140748 4729 ts_tablet_manager.cc:1403] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.228s user 0.017s sys 0.010s
I20260430 19:58:45.141021 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238e99c55dd6480da589cf3f43483773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.147647 4654 tablet_replica.cc:333] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:45.148416 4654 raft_consensus.cc:2243] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.148995 4654 raft_consensus.cc:2272] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.148552 4729 raft_consensus.cc:359] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.149312 4729 raft_consensus.cc:385] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.149648 4729 raft_consensus.cc:740] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.153131 4737 ts_tablet_manager.cc:1929] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.153563 4737 log.cc:1199] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/e42a2e81e4a44cc29f352d8d55a03c4a
I20260430 19:58:45.153844 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet e42a2e81e4a44cc29f352d8d55a03c4a (table default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]) successfully deleted
I20260430 19:58:45.154364 4737 ts_tablet_manager.cc:1950] T e42a2e81e4a44cc29f352d8d55a03c4a P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.130386 4023 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet d0f228d11b9641589825bb6a61310672 (table default.Table_5ea41a3ae89e4e45b01d9a69e437c571 [id=50df359f50de4b88a31deb7922615f33]). Expected version 2 got 0
I20260430 19:58:45.159327 4223 raft_consensus.cc:2468] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:45.161221 4765 consensus_queue.cc:1048] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:58:45.174413 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 76073435ab7949a0bb0606e6efaa48da (DEFAULT_TABLE table=default.Table_ace9ebd59ec0464cbca1f928a48eedaa [id=b75f16c3c90c4b6bb561726761e3077d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.175673 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76073435ab7949a0bb0606e6efaa48da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.156579 4729 consensus_queue.cc:260] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.183522 4742 ts_tablet_manager.cc:1929] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.183960 4742 log.cc:1199] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/d1d52994ff694886b7d4ff4224367082
I20260430 19:58:45.184617 4742 ts_tablet_manager.cc:1950] T d1d52994ff694886b7d4ff4224367082 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:45.189971 4654 ts_tablet_manager.cc:1916] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.208932 4512 tablet_service.cc:1511] Processing CreateTablet for tablet 76073435ab7949a0bb0606e6efaa48da (DEFAULT_TABLE table=default.Table_ace9ebd59ec0464cbca1f928a48eedaa [id=b75f16c3c90c4b6bb561726761e3077d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.210491 4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76073435ab7949a0bb0606e6efaa48da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.213264 4729 ts_tablet_manager.cc:1434] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.072s user 0.008s sys 0.000s
I20260430 19:58:45.212565 4737 tablet_replica.cc:333] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:45.215358 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet e42a2e81e4a44cc29f352d8d55a03c4a (table default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]) successfully deleted
I20260430 19:58:45.215845 4737 raft_consensus.cc:2243] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.216310 4737 raft_consensus.cc:2272] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.218127 4737 ts_tablet_manager.cc:1916] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:45.225493 4470 consensus_peers.cc:597] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Couldn't send request to peer d8345ec7f57c4399a8a4afe4dd43439e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6108085995074ea890cc6933bf45661f. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:45.242313 4362 tablet_service.cc:1511] Processing CreateTablet for tablet 2af571a9ee484ae5944d787def240c6e (DEFAULT_TABLE table=default.Table_ce9a7d51fd524fa6a8930979d0c5b125 [id=49546fb9471b4a1c969f4b438ec6ef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.243602 4362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2af571a9ee484ae5944d787def240c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.252439 4737 ts_tablet_manager.cc:1929] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.252837 4737 log.cc:1199] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/6856b41ce4ae450ca7b08015666c3a2d
I20260430 19:58:45.256006 4737 ts_tablet_manager.cc:1950] T 6856b41ce4ae450ca7b08015666c3a2d P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.258659 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 6856b41ce4ae450ca7b08015666c3a2d (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:45.261819 4737 tablet_replica.cc:333] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:45.263160 4737 raft_consensus.cc:2243] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:45.263305 4021 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet ae3cdfc0d2964360a431663e27fc6a14
I20260430 19:58:45.265980 4737 raft_consensus.cc:2272] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.268016 4737 ts_tablet_manager.cc:1916] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:45.268553 4027 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 76073435ab7949a0bb0606e6efaa48da consensus state has no leader
I20260430 19:58:45.283979 4654 ts_tablet_manager.cc:1929] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:45.284349 4654 log.cc:1199] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/6a7a82e60c974997b52a26650b131fc0
I20260430 19:58:45.298241 4729 tablet_bootstrap.cc:492] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:45.303874 4658 tablet_bootstrap.cc:492] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:45.305088 4729 tablet_bootstrap.cc:654] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.309324 4658 tablet_bootstrap.cc:654] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.313954 4654 ts_tablet_manager.cc:1950] T 6a7a82e60c974997b52a26650b131fc0 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:45.316018 4654 tablet_replica.cc:333] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:45.316658 4654 raft_consensus.cc:2243] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.318267 4737 ts_tablet_manager.cc:1929] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:45.322701 4654 raft_consensus.cc:2272] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.322837 4755 raft_consensus.cc:493] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:45.323491 4755 raft_consensus.cc:515] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:45.329347 4658 tablet_bootstrap.cc:492] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:45.329782 4658 ts_tablet_manager.cc:1403] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.000s
W20260430 19:58:45.332231 4158 leader_election.cc:343] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:45.332738 4658 raft_consensus.cc:359] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.333379 4658 raft_consensus.cc:385] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.333642 4658 raft_consensus.cc:740] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.334429 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 238e99c55dd6480da589cf3f43483773 (DEFAULT_TABLE table=default.Table_0dcf8ddbe6eb490b819da1d1cbc96508 [id=b43a4d5da8c84baa89c5eefa1fbd8409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.334370 4658 consensus_queue.cc:260] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.335263 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 2af571a9ee484ae5944d787def240c6e (DEFAULT_TABLE table=default.Table_ce9a7d51fd524fa6a8930979d0c5b125 [id=49546fb9471b4a1c969f4b438ec6ef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.335639 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238e99c55dd6480da589cf3f43483773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.336334 4658 ts_tablet_manager.cc:1434] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260430 19:58:45.336346 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2af571a9ee484ae5944d787def240c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.337244 4658 tablet_bootstrap.cc:492] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:45.336393 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 43d96039a1d64306b04ee49a6b86e854 (DEFAULT_TABLE table=default.Table_4ffb858382fa495f8af004403e5ea2db [id=abddf7bc9a6c4a76aa323bf84a2cc21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.337445 4204 tablet_service.cc:1511] Processing CreateTablet for tablet 76073435ab7949a0bb0606e6efaa48da (DEFAULT_TABLE table=default.Table_ace9ebd59ec0464cbca1f928a48eedaa [id=b75f16c3c90c4b6bb561726761e3077d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.337965 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d96039a1d64306b04ee49a6b86e854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.338792 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76073435ab7949a0bb0606e6efaa48da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.343669 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aa58a597da45ca8c00d26a615e10bb" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:45.352293 4658 tablet_bootstrap.cc:654] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.355720 4681 consensus_queue.cc:1048] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:45.366916 4654 ts_tablet_manager.cc:1916] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:45.377566 4015 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_5e8fd88607e445828f7d27136d5116a9 [id=b75f16c3c90c4b6bb561726761e3077d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 76073435ab7949a0bb0606e6efaa48da consensus state has no leader
I20260430 19:58:45.329962 4729 tablet_bootstrap.cc:492] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:45.381417 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_d5d25ec643f44b2190b0473839ea4da1 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]) successfully deleted
I20260430 19:58:45.381592 4729 ts_tablet_manager.cc:1403] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.009s
I20260430 19:58:45.382181 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet d1d52994ff694886b7d4ff4224367082 (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:45.385210 4022 master_service.cc:954] ReplaceTablet: received request to replace tablet a8b08893b8a5417392dcb7bc957c938b from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:45.384895 4729 raft_consensus.cc:359] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.330235 4737 log.cc:1199] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/6a7a82e60c974997b52a26650b131fc0
I20260430 19:58:45.387851 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aa58a597da45ca8c00d26a615e10bb" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:45.388603 4382 raft_consensus.cc:2393] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate 5a7356e99e744525861e12c108cfa3fe in this term.
I20260430 19:58:45.399915 4201 tablet_service.cc:1511] Processing CreateTablet for tablet 41df32f61a7f412eac7059f589388dae (DEFAULT_TABLE table=default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.401119 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41df32f61a7f412eac7059f589388dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.387342 4737 ts_tablet_manager.cc:1950] T 6a7a82e60c974997b52a26650b131fc0 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.389853 4470 leader_election.cc:304] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:45.405229 4681 raft_consensus.cc:3060] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.407994 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 6a7a82e60c974997b52a26650b131fc0 (table default.Table_d5d25ec643f44b2190b0473839ea4da1 [id=2302a4f9fee54fa6acbd21e0f5bbd31f]) successfully deleted
I20260430 19:58:45.412681 4737 tablet_replica.cc:333] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:45.412882 4681 raft_consensus.cc:2749] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:45.413774 4737 raft_consensus.cc:2243] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:45.415277 4658 tablet_bootstrap.cc:492] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:45.389920 4755 leader_election.cc:290] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:45.415680 4658 ts_tablet_manager.cc:1403] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.079s user 0.011s sys 0.013s
I20260430 19:58:45.416981 4737 raft_consensus.cc:2272] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.418090 4658 raft_consensus.cc:359] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.418743 4658 raft_consensus.cc:385] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.419009 4658 raft_consensus.cc:740] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.397799 4512 tablet_service.cc:1511] Processing CreateTablet for tablet 2af571a9ee484ae5944d787def240c6e (DEFAULT_TABLE table=default.Table_ce9a7d51fd524fa6a8930979d0c5b125 [id=49546fb9471b4a1c969f4b438ec6ef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.420986 4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2af571a9ee484ae5944d787def240c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.421206 4364 tablet_service.cc:1511] Processing CreateTablet for tablet 41df32f61a7f412eac7059f589388dae (DEFAULT_TABLE table=default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.422365 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41df32f61a7f412eac7059f589388dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.422343 4737 ts_tablet_manager.cc:1916] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.423799 4658 consensus_queue.cc:260] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.426272 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 41df32f61a7f412eac7059f589388dae (DEFAULT_TABLE table=default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:45.385517 4729 raft_consensus.cc:385] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.428983 4658 ts_tablet_manager.cc:1434] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260430 19:58:45.429849 4658 tablet_bootstrap.cc:492] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:45.431646 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41df32f61a7f412eac7059f589388dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:45.427965 4729 raft_consensus.cc:740] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.433929 4729 consensus_queue.cc:260] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.436802 4658 tablet_bootstrap.cc:654] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.439493 4729 ts_tablet_manager.cc:1434] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.057s user 0.003s sys 0.003s
I20260430 19:58:45.441054 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_276bd93958054b6b9ad30f314bbb487c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:45.443398 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_98329467d88d497ea88dc757f92d2aab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:45.444736 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98329467d88d497ea88dc757f92d2aab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:45.447309 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_276bd93958054b6b9ad30f314bbb487c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:45.447721 4658 tablet_bootstrap.cc:492] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:45.448092 4658 ts_tablet_manager.cc:1403] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.003s
I20260430 19:58:45.449043 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_8807d5c56dd74765bbf890fd151101ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:45.450215 4658 raft_consensus.cc:359] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:45.450574 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8807d5c56dd74765bbf890fd151101ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:45.450871 4658 raft_consensus.cc:385] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.451110 4658 raft_consensus.cc:740] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.451699 4658 consensus_queue.cc:260] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.453408 4658 ts_tablet_manager.cc:1434] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260430 19:58:45.454679 4020 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4ffb858382fa495f8af004403e5ea2db" } new_table_name: "default.Table_8a8d9de681864a819a452983f4fc1ee0" modify_external_catalogs: true
I20260430 19:58:45.456954 4749 raft_consensus.cc:493] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:45.457329 4749 raft_consensus.cc:515] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.459106 4749 leader_election.cc:290] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:45.461042 4658 tablet_bootstrap.cc:492] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:45.462832 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d96039a1d64306b04ee49a6b86e854" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:45.464623 4320 leader_election.cc:343] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:45.460587 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d96039a1d64306b04ee49a6b86e854" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:45.469555 4531 raft_consensus.cc:2468] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:45.470783 4317 leader_election.cc:304] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:45.471871 4749 raft_consensus.cc:2804] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:45.472179 4749 raft_consensus.cc:493] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:45.472476 4749 raft_consensus.cc:3060] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.479094 4749 raft_consensus.cc:515] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.484663 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d96039a1d64306b04ee49a6b86e854" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:45.485718 4531 raft_consensus.cc:3060] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.489898 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d96039a1d64306b04ee49a6b86e854" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
W20260430 19:58:45.495797 4320 leader_election.cc:343] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:45.500200 4531 raft_consensus.cc:2468] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:45.503468 4317 leader_election.cc:304] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:45.504282 4724 raft_consensus.cc:2804] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:45.504719 4724 raft_consensus.cc:697] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:45.507208 4729 tablet_bootstrap.cc:492] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:45.505453 4724 consensus_queue.cc:237] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.511943 4749 leader_election.cc:290] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:45.522296 4658 tablet_bootstrap.cc:654] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.536507 4737 ts_tablet_manager.cc:1929] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:45.536908 4737 log.cc:1199] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/d1d52994ff694886b7d4ff4224367082
I20260430 19:58:45.537628 4737 ts_tablet_manager.cc:1950] T d1d52994ff694886b7d4ff4224367082 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.538863 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet d1d52994ff694886b7d4ff4224367082 (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:45.544036 4729 tablet_bootstrap.cc:654] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.545905 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet d1d52994ff694886b7d4ff4224367082 with delete_type TABLET_DATA_DELETED (replaced by tablet c78d50690da7428fadb848dadefec47c) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:45.547953 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 6108085995074ea890cc6933bf45661f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:45.549799 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 504523705ff944c7b3f84db0ddf9f455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:45.550254 4654 ts_tablet_manager.cc:1929] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:45.552516 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a7a82e60c974997b52a26650b131fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:45.558174 4654 log.cc:1199] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/d1d52994ff694886b7d4ff4224367082
I20260430 19:58:45.559311 4654 ts_tablet_manager.cc:1950] T d1d52994ff694886b7d4ff4224367082 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:45.560722 4776 tablet_bootstrap.cc:492] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:45.563288 4737 tablet_replica.cc:333] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
W20260430 19:58:45.563714 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet d1d52994ff694886b7d4ff4224367082 because the tablet was not found. No further retry: Not found: Tablet not found: d1d52994ff694886b7d4ff4224367082
I20260430 19:58:45.564031 4737 raft_consensus.cc:2243] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:45.557837 4658 tablet_bootstrap.cc:492] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:45.564908 4737 raft_consensus.cc:2272] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.565341 4658 ts_tablet_manager.cc:1403] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.111s user 0.014s sys 0.001s
I20260430 19:58:45.568545 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet d1d52994ff694886b7d4ff4224367082 (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]) successfully deleted
I20260430 19:58:45.569617 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:45.571134 4026 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.3:35079 (request call id 12) took 1099 ms (client timeout 1000 ms). Trace:
I20260430 19:58:45.572786 4658 raft_consensus.cc:359] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.582881 4658 raft_consensus.cc:385] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.583168 4658 raft_consensus.cc:740] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.581099 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_0dcf8ddbe6eb490b819da1d1cbc96508" } modify_external_catalogs: true
I20260430 19:58:45.585156 4737 ts_tablet_manager.cc:1916] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.585070 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_0dcf8ddbe6eb490b819da1d1cbc96508" } modify_external_catalogs: true
I20260430 19:58:45.586956 4658 consensus_queue.cc:260] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.587700 4776 tablet_bootstrap.cc:654] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:45.584302 4020 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43d96039a1d64306b04ee49a6b86e854 consensus state has no leader
W20260430 19:58:45.593884 4026 rpcz_store.cc:269] 0430 19:58:44.471706 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:44.573306 (+101600us) service_pool.cc:225] Handling call
0430 19:58:44.623588 (+ 50282us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:45.571065 (+947477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3100,"reported_tablets":5,"spinlock_wait_cycles":84126976}
I20260430 19:58:45.596997 4658 ts_tablet_manager.cc:1434] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260430 19:58:45.603996 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet a8b08893b8a5417392dcb7bc957c938b
I20260430 19:58:45.605989 4022 catalog_manager.cc:6741] ReplaceTablet: tablet a8b08893b8a5417392dcb7bc957c938b deleted and replaced by tablet a4b0d830f1b14ce382b2e0242f6565e7
I20260430 19:58:45.606997 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet a8b08893b8a5417392dcb7bc957c938b with delete_type TABLET_DATA_DELETED (replaced by tablet a4b0d830f1b14ce382b2e0242f6565e7) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:45.607759 4654 tablet_replica.cc:333] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:45.608067 4201 tablet_service.cc:1558] Processing DeleteTablet for tablet a8b08893b8a5417392dcb7bc957c938b with delete_type TABLET_DATA_DELETED (replaced by tablet a4b0d830f1b14ce382b2e0242f6565e7) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:45.608635 4654 raft_consensus.cc:2243] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.608839 4742 tablet_replica.cc:333] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:45.609268 4654 raft_consensus.cc:2272] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.609587 4742 raft_consensus.cc:2243] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.610023 4742 raft_consensus.cc:2272] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.610674 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet a8b08893b8a5417392dcb7bc957c938b with delete_type TABLET_DATA_DELETED (replaced by tablet a4b0d830f1b14ce382b2e0242f6565e7) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:45.611056 4654 ts_tablet_manager.cc:1916] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.611891 4742 ts_tablet_manager.cc:1916] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.617903 4658 tablet_bootstrap.cc:492] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:45.624697 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_e4bcca1db8ad434780968ac6fe236a2c" } new_table_name: "default.Table_936e51d7cb7f45d08b267025523f49a3" modify_external_catalogs: true
I20260430 19:58:45.633571 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_4b6fc137cf6945079fa8310db94a2b1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:45.638423 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b6fc137cf6945079fa8310db94a2b1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:45.638988 4729 tablet_bootstrap.cc:492] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:45.639431 4729 ts_tablet_manager.cc:1403] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.132s user 0.019s sys 0.010s
I20260430 19:58:45.644716 4658 tablet_bootstrap.cc:654] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.642496 4729 raft_consensus.cc:359] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.650242 4729 raft_consensus.cc:385] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.650537 4729 raft_consensus.cc:740] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.651489 4729 consensus_queue.cc:260] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.652693 4654 ts_tablet_manager.cc:1929] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.655186 4729 ts_tablet_manager.cc:1434] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260430 19:58:45.657029 4729 tablet_bootstrap.cc:492] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:45.662825 4654 log.cc:1199] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/a8b08893b8a5417392dcb7bc957c938b
I20260430 19:58:45.664230 4654 ts_tablet_manager.cc:1950] T a8b08893b8a5417392dcb7bc957c938b P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:45.666237 4729 tablet_bootstrap.cc:654] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.658816 4658 tablet_bootstrap.cc:492] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:45.666817 4658 ts_tablet_manager.cc:1403] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.049s user 0.009s sys 0.001s
I20260430 19:58:45.669173 4658 raft_consensus.cc:359] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.670475 4658 raft_consensus.cc:385] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.671106 4658 raft_consensus.cc:740] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.672380 4776 tablet_bootstrap.cc:492] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:45.672847 4776 ts_tablet_manager.cc:1403] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.117s user 0.004s sys 0.009s
I20260430 19:58:45.673053 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet a8b08893b8a5417392dcb7bc957c938b (table default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]) successfully deleted
I20260430 19:58:45.674276 4658 consensus_queue.cc:260] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.676460 4776 raft_consensus.cc:359] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.677160 4776 raft_consensus.cc:385] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.677486 4776 raft_consensus.cc:740] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.678733 4729 tablet_bootstrap.cc:492] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:45.680452 4729 ts_tablet_manager.cc:1403] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.007s
I20260430 19:58:45.678850 4776 consensus_queue.cc:260] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.687971 4742 ts_tablet_manager.cc:1929] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.690435 4737 ts_tablet_manager.cc:1929] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:45.690672 4658 ts_tablet_manager.cc:1434] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260430 19:58:45.691162 4742 log.cc:1199] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/a8b08893b8a5417392dcb7bc957c938b
I20260430 19:58:45.691876 4742 ts_tablet_manager.cc:1950] T a8b08893b8a5417392dcb7bc957c938b P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:45.692257 4737 log.cc:1199] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/6108085995074ea890cc6933bf45661f
I20260430 19:58:45.693006 4737 ts_tablet_manager.cc:1950] T 6108085995074ea890cc6933bf45661f P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.689929 4776 ts_tablet_manager.cc:1434] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260430 19:58:45.697664 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet a8b08893b8a5417392dcb7bc957c938b (table default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]) successfully deleted
I20260430 19:58:45.696532 4776 tablet_bootstrap.cc:492] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:45.700763 4724 raft_consensus.cc:493] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:45.701218 4724 raft_consensus.cc:515] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.703104 4724 leader_election.cc:290] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:45.704579 4776 tablet_bootstrap.cc:654] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.706405 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76073435ab7949a0bb0606e6efaa48da" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:45.708323 4320 leader_election.cc:343] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:45.690526 4729 raft_consensus.cc:359] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.704802 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76073435ab7949a0bb0606e6efaa48da" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:45.709136 4729 raft_consensus.cc:385] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.709367 4531 raft_consensus.cc:2468] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:45.709422 4729 raft_consensus.cc:740] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.710330 4729 consensus_queue.cc:260] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.710719 4317 leader_election.cc:304] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:45.717708 4724 raft_consensus.cc:2804] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:45.719300 4729 ts_tablet_manager.cc:1434] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260430 19:58:45.720283 4737 tablet_replica.cc:333] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:45.720984 4737 raft_consensus.cc:2243] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.723421 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 6108085995074ea890cc6933bf45661f (table default.Table_78209fb594f841308f82995e31eff165 [id=f509dee5b07f496381f8a0edcbb50ced]) successfully deleted
I20260430 19:58:45.722368 4027 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_8a8d9de681864a819a452983f4fc1ee0 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]). Expected version 2 got 0
I20260430 19:58:45.724334 4724 raft_consensus.cc:493] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:45.724934 4729 tablet_bootstrap.cc:492] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:45.726493 4724 raft_consensus.cc:3060] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.730659 4737 raft_consensus.cc:2272] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.732405 4737 ts_tablet_manager.cc:1916] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.739535 4729 tablet_bootstrap.cc:654] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.750660 4729 tablet_bootstrap.cc:492] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:45.751111 4729 ts_tablet_manager.cc:1403] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.002s
I20260430 19:58:45.751945 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet 504523705ff944c7b3f84db0ddf9f455 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:45.758440 4776 tablet_bootstrap.cc:492] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:45.758962 4776 ts_tablet_manager.cc:1403] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.002s
I20260430 19:58:45.761518 4776 raft_consensus.cc:359] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.763427 4724 raft_consensus.cc:515] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.767123 4737 ts_tablet_manager.cc:1929] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.767483 4737 log.cc:1199] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/504523705ff944c7b3f84db0ddf9f455
I20260430 19:58:45.768065 4737 ts_tablet_manager.cc:1950] T 504523705ff944c7b3f84db0ddf9f455 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.769161 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76073435ab7949a0bb0606e6efaa48da" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:45.770656 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 504523705ff944c7b3f84db0ddf9f455 (table default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e [id=0f3d9fa7646043459be13fcce244dc0f]) successfully deleted
W20260430 19:58:45.771389 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6a7a82e60c974997b52a26650b131fc0 because the tablet was not found. No further retry: Not found: Tablet not found: 6a7a82e60c974997b52a26650b131fc0
W20260430 19:58:45.771611 4320 leader_election.cc:343] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:45.772115 4776 raft_consensus.cc:385] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260430 19:58:45.772495 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet e42a2e81e4a44cc29f352d8d55a03c4a because the tablet was not found. No further retry: Not found: Tablet not found: e42a2e81e4a44cc29f352d8d55a03c4a
I20260430 19:58:45.773600 4729 raft_consensus.cc:359] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.774564 4776 raft_consensus.cc:740] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.775382 4776 consensus_queue.cc:260] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.780170 4749 raft_consensus.cc:493] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:45.780341 4776 ts_tablet_manager.cc:1434] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.021s user 0.001s sys 0.004s
I20260430 19:58:45.774226 4729 raft_consensus.cc:385] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.780781 4729 raft_consensus.cc:740] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.780637 4749 raft_consensus.cc:515] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.781518 4729 consensus_queue.cc:260] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.782022 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_95bf05b38f054143a0f537b967e00b93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:45.784026 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95bf05b38f054143a0f537b967e00b93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:45.785125 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238e99c55dd6480da589cf3f43483773" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:45.785712 4531 raft_consensus.cc:2468] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:45.785771 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238e99c55dd6480da589cf3f43483773" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:45.786346 4223 raft_consensus.cc:2468] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:45.787060 4317 leader_election.cc:304] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:45.787804 4744 raft_consensus.cc:2804] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:45.782428 4749 leader_election.cc:290] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:45.788148 4744 raft_consensus.cc:493] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:45.788376 4776 tablet_bootstrap.cc:492] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:45.788471 4744 raft_consensus.cc:3060] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.791123 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76073435ab7949a0bb0606e6efaa48da" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:45.791651 4531 raft_consensus.cc:3060] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.795945 4729 ts_tablet_manager.cc:1434] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.045s user 0.005s sys 0.001s
I20260430 19:58:45.797235 4744 raft_consensus.cc:515] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.797850 4021 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_8a8d9de681864a819a452983f4fc1ee0" } new_table_name: "default.Table_b8df0c78d8ee4cc29163092181f1e028" modify_external_catalogs: true
I20260430 19:58:45.799852 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238e99c55dd6480da589cf3f43483773" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:45.800424 4223 raft_consensus.cc:3060] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.807065 4729 tablet_bootstrap.cc:492] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:45.807219 4223 raft_consensus.cc:2468] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:45.808386 4320 leader_election.cc:304] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:45.811813 4749 raft_consensus.cc:2804] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:45.813898 4776 tablet_bootstrap.cc:654] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.815147 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "238e99c55dd6480da589cf3f43483773" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:45.815698 4532 raft_consensus.cc:3060] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:45.818677 4744 leader_election.cc:290] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:45.818938 4737 tablet_replica.cc:333] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:45.819694 4737 raft_consensus.cc:2243] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:45.820327 4737 pending_rounds.cc:70] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Trying to abort 1 pending ops.
I20260430 19:58:45.820736 4737 pending_rounds.cc:77] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964096314658816 op_type: NO_OP noop_request { }
I20260430 19:58:45.821069 4737 raft_consensus.cc:2889] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:45.821357 4737 raft_consensus.cc:2272] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.823807 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_b8cbfa7ae3844f47b3e6606218439a88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:45.824476 4737 ts_tablet_manager.cc:1916] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.825233 4532 raft_consensus.cc:2468] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
W20260430 19:58:45.825321 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8cbfa7ae3844f47b3e6606218439a88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:45.833776 4729 tablet_bootstrap.cc:654] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.835860 4531 raft_consensus.cc:2468] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:45.847519 4026 master_service.cc:954] ReplaceTablet: received request to replace tablet d7aa58a597da45ca8c00d26a615e10bb from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:45.852378 4724 leader_election.cc:290] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:45.854092 4317 leader_election.cc:304] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:45.855309 4744 raft_consensus.cc:2804] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:45.855736 4744 raft_consensus.cc:697] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:45.856541 4744 consensus_queue.cc:237] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.849961 4737 ts_tablet_manager.cc:1929] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:45.851840 4729 tablet_bootstrap.cc:492] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:45.857470 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_0a03c960230b4206944b333b146c39c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:45.858163 4729 ts_tablet_manager.cc:1403] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.051s user 0.011s sys 0.002s
I20260430 19:58:45.860602 4729 raft_consensus.cc:359] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.861207 4729 raft_consensus.cc:385] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.861452 4729 raft_consensus.cc:740] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.862125 4729 consensus_queue.cc:260] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:45.869972 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a03c960230b4206944b333b146c39c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:45.872553 4749 raft_consensus.cc:697] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:45.873214 4729 ts_tablet_manager.cc:1434] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260430 19:58:45.873760 4737 log.cc:1199] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/a8b08893b8a5417392dcb7bc957c938b
I20260430 19:58:45.873546 4749 consensus_queue.cc:237] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.874469 4737 ts_tablet_manager.cc:1950] T a8b08893b8a5417392dcb7bc957c938b P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:45.877599 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet a8b08893b8a5417392dcb7bc957c938b (table default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]) successfully deleted
I20260430 19:58:45.880577 4776 tablet_bootstrap.cc:492] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:45.883013 4776 ts_tablet_manager.cc:1403] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.095s user 0.008s sys 0.012s
I20260430 19:58:45.885499 4776 raft_consensus.cc:359] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.890295 4776 raft_consensus.cc:385] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.890669 4776 raft_consensus.cc:740] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.891369 4776 consensus_queue.cc:260] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.895306 4776 ts_tablet_manager.cc:1434] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.012s user 0.003s sys 0.002s
I20260430 19:58:45.896489 4776 tablet_bootstrap.cc:492] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:45.904615 4776 tablet_bootstrap.cc:654] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.915731 4776 tablet_bootstrap.cc:492] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:45.916263 4776 ts_tablet_manager.cc:1403] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.008s
I20260430 19:58:45.918745 4776 raft_consensus.cc:359] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.919516 4776 raft_consensus.cc:385] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:45.919838 4776 raft_consensus.cc:740] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:45.920593 4776 consensus_queue.cc:260] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:45.928090 4776 ts_tablet_manager.cc:1434] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.011s user 0.003s sys 0.001s
I20260430 19:58:45.931438 4776 tablet_bootstrap.cc:492] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:45.937168 4776 tablet_bootstrap.cc:654] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:45.970785 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet d7aa58a597da45ca8c00d26a615e10bb
I20260430 19:58:45.973064 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet d7aa58a597da45ca8c00d26a615e10bb with delete_type TABLET_DATA_DELETED (replaced by tablet 404f20adf9d741f3a036b06cbdbaedb8) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:45.973757 4737 tablet_replica.cc:333] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:45.974020 4026 catalog_manager.cc:6741] ReplaceTablet: tablet d7aa58a597da45ca8c00d26a615e10bb deleted and replaced by tablet 404f20adf9d741f3a036b06cbdbaedb8
I20260430 19:58:45.974419 4737 raft_consensus.cc:2243] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.975013 4737 raft_consensus.cc:2272] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.976287 4364 tablet_service.cc:1558] Processing DeleteTablet for tablet d7aa58a597da45ca8c00d26a615e10bb with delete_type TABLET_DATA_DELETED (replaced by tablet 404f20adf9d741f3a036b06cbdbaedb8) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:45.976536 4737 ts_tablet_manager.cc:1916] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.976928 4201 tablet_service.cc:1558] Processing DeleteTablet for tablet d7aa58a597da45ca8c00d26a615e10bb with delete_type TABLET_DATA_DELETED (replaced by tablet 404f20adf9d741f3a036b06cbdbaedb8) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:45.977353 4020 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_5e8fd88607e445828f7d27136d5116a9 [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 1 got 0
I20260430 19:58:45.977638 4654 tablet_replica.cc:333] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:45.978313 4654 raft_consensus.cc:2243] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:45.978751 4654 raft_consensus.cc:2272] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.978413 4020 catalog_manager.cc:5671] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2). New cstate: current_term: 1 leader_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:45.979727 4020 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_8a8d9de681864a819a452983f4fc1ee0 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]). Expected version 2 got 0
I20260430 19:58:45.979883 4742 tablet_replica.cc:333] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:45.980620 4742 raft_consensus.cc:2243] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Raft consensus shutting down.
W20260430 19:58:45.980877 4021 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43d96039a1d64306b04ee49a6b86e854 consensus state has no leader
I20260430 19:58:45.981298 4742 pending_rounds.cc:70] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Trying to abort 1 pending ops.
I20260430 19:58:45.981527 4742 pending_rounds.cc:77] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964096622182400 op_type: NO_OP noop_request { }
I20260430 19:58:45.981876 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet d0f228d11b9641589825bb6a61310672 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:45.984400 4654 ts_tablet_manager.cc:1916] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.981863 4742 raft_consensus.cc:2889] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:45.986869 4742 raft_consensus.cc:2272] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:45.987221 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_05824f167de14d4bbfc98363345b38ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:45.988863 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05824f167de14d4bbfc98363345b38ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:45.990028 4742 ts_tablet_manager.cc:1916] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:45.993323 4737 ts_tablet_manager.cc:1929] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:45.996531 4737 log.cc:1199] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/d7aa58a597da45ca8c00d26a615e10bb
I20260430 19:58:45.997567 4737 ts_tablet_manager.cc:1950] T d7aa58a597da45ca8c00d26a615e10bb P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:46.000617 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet d7aa58a597da45ca8c00d26a615e10bb (table default.Table_e4bcca1db8ad434780968ac6fe236a2c [id=06859fba4a5d4b8f978e6f56edc5fb85]) successfully deleted
I20260430 19:58:46.013912 4029 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_5e8fd88607e445828f7d27136d5116a9 [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 1 got 0
I20260430 19:58:46.014698 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 238e99c55dd6480da589cf3f43483773
W20260430 19:58:46.014865 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 404f20adf9d741f3a036b06cbdbaedb8 has no consensus state
I20260430 19:58:46.014807 4201 tablet_service.cc:1511] Processing CreateTablet for tablet 3946fffc0d2f47f789c7341dc66d603e (DEFAULT_TABLE table=default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.015867 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3946fffc0d2f47f789c7341dc66d603e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.018365 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet d0f228d11b9641589825bb6a61310672
I20260430 19:58:46.019829 4025 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:55130 (request call id 60) took 1358 ms. Trace:
I20260430 19:58:46.025285 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet d0f228d11b9641589825bb6a61310672 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:44 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.025303 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 238e99c55dd6480da589cf3f43483773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:45 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.025993 4737 tablet_replica.cc:333] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:46.026714 4737 raft_consensus.cc:2243] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:46.027199 4737 raft_consensus.cc:2272] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:46.030151 4737 ts_tablet_manager.cc:1916] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.039481 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 238e99c55dd6480da589cf3f43483773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:45 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.040122 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet d0f228d11b9641589825bb6a61310672 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:44 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.042150 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet d1d52994ff694886b7d4ff4224367082 with delete_type TABLET_DATA_DELETED (replaced by tablet c78d50690da7428fadb848dadefec47c) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.042820 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet 6108085995074ea890cc6933bf45661f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.043414 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 504523705ff944c7b3f84db0ddf9f455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.043782 4737 ts_tablet_manager.cc:1929] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:46.043951 4511 tablet_service.cc:1558] Processing DeleteTablet for tablet a8b08893b8a5417392dcb7bc957c938b with delete_type TABLET_DATA_DELETED (replaced by tablet a4b0d830f1b14ce382b2e0242f6565e7) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.046744 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a7a82e60c974997b52a26650b131fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.047384 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet d1d52994ff694886b7d4ff4224367082 with delete_type TABLET_DATA_DELETED (replaced by tablet c78d50690da7428fadb848dadefec47c) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.044134 4737 log.cc:1199] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/d0f228d11b9641589825bb6a61310672
I20260430 19:58:46.048434 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet 504523705ff944c7b3f84db0ddf9f455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.048528 4737 ts_tablet_manager.cc:1950] T d0f228d11b9641589825bb6a61310672 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:46.050271 4362 tablet_service.cc:1558] Processing DeleteTablet for tablet d1d52994ff694886b7d4ff4224367082 with delete_type TABLET_DATA_DELETED (replaced by tablet c78d50690da7428fadb848dadefec47c) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.050654 4364 tablet_service.cc:1558] Processing DeleteTablet for tablet 238e99c55dd6480da589cf3f43483773 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:45 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.050751 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.050935 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet a8b08893b8a5417392dcb7bc957c938b with delete_type TABLET_DATA_DELETED (replaced by tablet a4b0d830f1b14ce382b2e0242f6565e7) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.051258 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet d0f228d11b9641589825bb6a61310672 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:44 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.052078 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a7a82e60c974997b52a26650b131fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.053373 4359 tablet_service.cc:1558] Processing DeleteTablet for tablet 504523705ff944c7b3f84db0ddf9f455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.054044 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet a8b08893b8a5417392dcb7bc957c938b with delete_type TABLET_DATA_DELETED (replaced by tablet a4b0d830f1b14ce382b2e0242f6565e7) from {username='slave'} at 127.0.0.1:41996
W20260430 19:58:46.028419 4023 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.2:49905 (request call id 20) took 1464 ms (client timeout 1000 ms). Trace:
I20260430 19:58:46.054783 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 6108085995074ea890cc6933bf45661f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.055611 4737 tablet_replica.cc:333] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:46.056241 4737 raft_consensus.cc:2243] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:46.056653 4737 raft_consensus.cc:2272] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:46.063823 4737 ts_tablet_manager.cc:1916] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.020147 4025 rpcz_store.cc:276] 0430 19:58:44.661852 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:44.662673 (+ 821us) service_pool.cc:225] Handling call
0430 19:58:44.687378 (+ 24705us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f2929117c46 00007f2926d29024 00007f2926d28fb2 00007f2926d28ee1 00007f2926d28e5c 00007f2926d28112 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:44.688946 (+ 1568us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0430 19:58:44.690006 (+ 1060us) catalog_manager.cc:2815] Modifying in-memory table state
0430 19:58:44.690148 (+ 142us) catalog_manager.cc:2824] Locking tablets
0430 19:58:45.533895 (+843747us) catalog_manager.cc:2838] Removing table and tablets from system table
0430 19:58:45.716073 (+182178us) catalog_manager.cc:2856] Removing table from by-name map
0430 19:58:45.716312 (+ 239us) catalog_manager.cc:2872] Committing in-memory state
0430 19:58:46.018261 (+301949us) catalog_manager.cc:2876] Aborting table tasks
0430 19:58:46.019769 (+ 1508us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:45.666733 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:45.667125 (+ 392us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:45.667150 (+ 25us) write_op.cc:435] Acquired schema lock
0430 19:58:45.667163 (+ 13us) tablet.cc:662] Decoding operations
0430 19:58:45.667469 (+ 306us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:45.667507 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:45.667528 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0430 19:58:45.674685 (+ 7157us) tablet.cc:701] Row locks acquired
0430 19:58:45.674702 (+ 17us) write_op.cc:260] PREPARE: finished
0430 19:58:45.674831 (+ 129us) write_op.cc:270] Start()
0430 19:58:45.674922 (+ 91us) write_op.cc:276] Timestamp: P: 1777579125674771 usec, L: 0
0430 19:58:45.674938 (+ 16us) op_driver.cc:348] REPLICATION: starting
0430 19:58:45.675417 (+ 479us) log.cc:844] Serialized 789 byte log entry
0430 19:58:45.714209 (+ 38792us) op_driver.cc:464] REPLICATION: finished
0430 19:58:45.714804 (+ 595us) write_op.cc:301] APPLY: starting
0430 19:58:45.714860 (+ 56us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:45.714961 (+ 101us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:45.715054 (+ 93us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:45.715262 (+ 208us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:45.715402 (+ 140us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0430 19:58:45.715441 (+ 39us) write_op.cc:312] APPLY: finished
0430 19:58:45.715614 (+ 173us) log.cc:844] Serialized 28 byte log entry
0430 19:58:45.715763 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:45.715827 (+ 64us) write_op.cc:454] Released schema lock
0430 19:58:45.715857 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:45.716081 (+ 224us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:45.716104 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":55068928,"child_traces":[["op",{"apply.queue_time_us":445,"apply.run_cpu_time_us":1250,"apply.run_wall_time_us":1250,"num_ops":2,"prepare.queue_time_us":125695,"prepare.run_cpu_time_us":2218,"prepare.run_wall_time_us":9954,"raft.queue_time_us":143,"raft.run_cpu_time_us":668,"raft.run_wall_time_us":669,"replication_time_us":39172}]]}
I20260430 19:58:46.053429 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_952e6a513f694f3ea455f5264631e8b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.070200 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_952e6a513f694f3ea455f5264631e8b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:46.054414 4023 rpcz_store.cc:269] 0430 19:58:44.563638 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:44.575703 (+ 12065us) service_pool.cc:225] Handling call
0430 19:58:44.615725 (+ 40022us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:46.028343 (+1412618us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:45.202129 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:45.202527 (+ 398us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:45.202550 (+ 23us) write_op.cc:435] Acquired schema lock
0430 19:58:45.202562 (+ 12us) tablet.cc:662] Decoding operations
0430 19:58:45.227286 (+ 24724us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:45.227328 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:45.227353 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:45.227508 (+ 155us) tablet.cc:701] Row locks acquired
0430 19:58:45.227521 (+ 13us) write_op.cc:260] PREPARE: finished
0430 19:58:45.227649 (+ 128us) write_op.cc:270] Start()
0430 19:58:45.227738 (+ 89us) write_op.cc:276] Timestamp: P: 1777579125227578 usec, L: 0
0430 19:58:45.227754 (+ 16us) op_driver.cc:348] REPLICATION: starting
0430 19:58:45.443418 (+215664us) log.cc:844] Serialized 526 byte log entry
0430 19:58:45.443987 (+ 569us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927268951 00007f29272fb208 00007f292776d9f1 00007f292776d1c8 00007f2927771e61 00007f2927771e19 00007f2927771da9 00007f2927771d71 00007f2927770f9d 00000000005b3bc4 00000000005b39f9
0430 19:58:45.528568 (+ 84581us) op_driver.cc:464] REPLICATION: finished
0430 19:58:45.528804 (+ 236us) write_op.cc:301] APPLY: starting
0430 19:58:45.528861 (+ 57us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:45.528949 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:45.528965 (+ 16us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:45.529123 (+ 158us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:45.529255 (+ 132us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:45.529295 (+ 40us) write_op.cc:312] APPLY: finished
0430 19:58:45.529461 (+ 166us) log.cc:844] Serialized 22 byte log entry
0430 19:58:45.529595 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:45.529650 (+ 55us) write_op.cc:454] Released schema lock
0430 19:58:45.529677 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:45.529892 (+ 215us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:45.529917 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9634,"reported_tablets":5,"spinlock_wait_cycles":88324864,"child_traces":[["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":1061,"apply.run_wall_time_us":1061,"num_ops":1,"prepare.queue_time_us":32399,"prepare.run_cpu_time_us":3804,"prepare.run_wall_time_us":242102,"raft.queue_time_us":10432,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":13028,"replication_time_us":300729,"spinlock_wait_cycles":494830976}]]}
I20260430 19:58:46.054906 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet d0f228d11b9641589825bb6a61310672 (table default.Table_5ea41a3ae89e4e45b01d9a69e437c571 [id=50df359f50de4b88a31deb7922615f33]) successfully deleted
W20260430 19:58:46.034579 4024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.1:59387 (request call id 24) took 1111 ms (client timeout 1000 ms). Trace:
I20260430 19:58:46.065389 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_1921e3a044f04ea8a445515ab59a8830"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.077069 4024 rpcz_store.cc:269] 0430 19:58:44.922925 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:44.923069 (+ 144us) service_pool.cc:225] Handling call
0430 19:58:46.034500 (+1111431us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6}
W20260430 19:58:46.078316 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1921e3a044f04ea8a445515ab59a8830 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.079070 4022 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_b8df0c78d8ee4cc29163092181f1e028 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]). Expected version 3 got 0
I20260430 19:58:46.079754 4022 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 41df32f61a7f412eac7059f589388dae (table default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 1 got 0
I20260430 19:58:46.080343 4022 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_5e8fd88607e445828f7d27136d5116a9 [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 1 got 0
I20260430 19:58:46.084864 4021 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 41df32f61a7f412eac7059f589388dae (table default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 1 got 0
I20260430 19:58:46.085599 4021 catalog_manager.cc:5671] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2). New cstate: current_term: 1 leader_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:46.087056 4021 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_5e8fd88607e445828f7d27136d5116a9 [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 1 got 0
I20260430 19:58:46.092454 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet e42a2e81e4a44cc29f352d8d55a03c4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:43 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.100543 4737 ts_tablet_manager.cc:1929] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:46.100883 4737 log.cc:1199] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/238e99c55dd6480da589cf3f43483773
I20260430 19:58:46.101468 4737 ts_tablet_manager.cc:1950] T 238e99c55dd6480da589cf3f43483773 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:46.105748 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 238e99c55dd6480da589cf3f43483773 (table default.Table_0dcf8ddbe6eb490b819da1d1cbc96508 [id=b43a4d5da8c84baa89c5eefa1fbd8409]) successfully deleted
W20260430 19:58:46.106451 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet d1d52994ff694886b7d4ff4224367082 because the tablet was not found. No further retry: Not found: Tablet not found: d1d52994ff694886b7d4ff4224367082
W20260430 19:58:46.107120 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6108085995074ea890cc6933bf45661f because the tablet was not found. No further retry: Not found: Tablet not found: 6108085995074ea890cc6933bf45661f
W20260430 19:58:46.108572 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 504523705ff944c7b3f84db0ddf9f455 because the tablet was not found. No further retry: Not found: Tablet not found: 504523705ff944c7b3f84db0ddf9f455
W20260430 19:58:46.109244 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet a8b08893b8a5417392dcb7bc957c938b because the tablet was not found. No further retry: Not found: Tablet not found: a8b08893b8a5417392dcb7bc957c938b
I20260430 19:58:46.046350 4654 ts_tablet_manager.cc:1929] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:46.114907 4654 log.cc:1199] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/d7aa58a597da45ca8c00d26a615e10bb
I20260430 19:58:46.115785 4654 ts_tablet_manager.cc:1950] T d7aa58a597da45ca8c00d26a615e10bb P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:46.117975 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet d7aa58a597da45ca8c00d26a615e10bb (table default.Table_936e51d7cb7f45d08b267025523f49a3 [id=06859fba4a5d4b8f978e6f56edc5fb85]) successfully deleted
I20260430 19:58:46.121061 4531 raft_consensus.cc:1275] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Refusing update from remote peer d8345ec7f57c4399a8a4afe4dd43439e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:46.121320 4223 raft_consensus.cc:3060] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.122653 4749 consensus_queue.cc:1048] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:46.132282 4223 raft_consensus.cc:1275] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Refusing update from remote peer d8345ec7f57c4399a8a4afe4dd43439e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:46.133838 4749 consensus_queue.cc:1048] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:46.143405 4742 ts_tablet_manager.cc:1929] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:46.143880 4742 log.cc:1199] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/d7aa58a597da45ca8c00d26a615e10bb
I20260430 19:58:46.144701 4742 ts_tablet_manager.cc:1950] T d7aa58a597da45ca8c00d26a615e10bb P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:46.151969 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet d7aa58a597da45ca8c00d26a615e10bb (table default.Table_936e51d7cb7f45d08b267025523f49a3 [id=06859fba4a5d4b8f978e6f56edc5fb85]) successfully deleted
I20260430 19:58:46.153400 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_7740c2a925e845839ee3fa21dc4b4553"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.159196 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7740c2a925e845839ee3fa21dc4b4553 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.161198 4028 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_b8df0c78d8ee4cc29163092181f1e028 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]). Expected version 3 got 0
I20260430 19:58:46.161445 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 3946fffc0d2f47f789c7341dc66d603e (DEFAULT_TABLE table=default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.162850 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3946fffc0d2f47f789c7341dc66d603e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.168795 4029 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_b8df0c78d8ee4cc29163092181f1e028 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]). Expected version 3 got 0
I20260430 19:58:46.179529 4802 tablet.cc:1724] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260430 19:58:46.186517 4750 raft_consensus.cc:493] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:46.186961 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5e8fd88607e445828f7d27136d5116a9" } new_table_name: "default.Table_7f29ad8fb22d4a378afb83449fab451b" modify_external_catalogs: true
I20260430 19:58:46.186977 4750 raft_consensus.cc:515] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.188750 4750 leader_election.cc:290] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:46.190325 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af571a9ee484ae5944d787def240c6e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:46.190351 4028 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_b8df0c78d8ee4cc29163092181f1e028 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]). Expected version 3 got 0
I20260430 19:58:46.191202 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af571a9ee484ae5944d787def240c6e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:46.191409 4382 raft_consensus.cc:2468] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:46.191803 4532 raft_consensus.cc:2468] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:46.192456 4161 leader_election.cc:304] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:46.194095 4750 raft_consensus.cc:2804] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:46.194432 4750 raft_consensus.cc:493] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:46.194748 4750 raft_consensus.cc:3060] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.195878 4801 tablet.cc:1724] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260430 19:58:46.201699 4750 raft_consensus.cc:515] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.204841 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af571a9ee484ae5944d787def240c6e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:46.205529 4382 raft_consensus.cc:3060] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.205494 4511 tablet_service.cc:1511] Processing CreateTablet for tablet 3946fffc0d2f47f789c7341dc66d603e (DEFAULT_TABLE table=default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.206673 4511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3946fffc0d2f47f789c7341dc66d603e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.206576 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af571a9ee484ae5944d787def240c6e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:46.207127 4532 raft_consensus.cc:3060] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.214563 4750 leader_election.cc:290] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:46.215761 4382 raft_consensus.cc:2468] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:46.217191 4161 leader_election.cc:304] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:46.218098 4750 raft_consensus.cc:2804] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:46.219352 4532 raft_consensus.cc:2468] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:46.230448 4359 tablet_service.cc:1511] Processing CreateTablet for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba (DEFAULT_TABLE table=default.Table_ccf1123d6d5d4e489838f28c6125aa15 [id=007139834a09457dbfcb9a426839d894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.231792 4359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:46.240489 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet d1d52994ff694886b7d4ff4224367082 because the tablet was not found. No further retry: Not found: Tablet not found: d1d52994ff694886b7d4ff4224367082
I20260430 19:58:46.246958 4654 tablet_replica.cc:333] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:46.247390 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba (DEFAULT_TABLE table=default.Table_ccf1123d6d5d4e489838f28c6125aa15 [id=007139834a09457dbfcb9a426839d894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.247752 4654 raft_consensus.cc:2243] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:46.248418 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.248524 4654 pending_rounds.cc:70] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:46.248824 4654 pending_rounds.cc:77] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964099593711616 op_type: NO_OP noop_request { }
I20260430 19:58:46.249233 4654 raft_consensus.cc:2889] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:46.249604 4654 raft_consensus.cc:2272] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:46.255503 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_b8df0c78d8ee4cc29163092181f1e028" } modify_external_catalogs: true
I20260430 19:58:46.256121 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_b8df0c78d8ee4cc29163092181f1e028" } modify_external_catalogs: true
I20260430 19:58:46.258005 4750 raft_consensus.cc:697] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:46.260844 4026 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_98329467d88d497ea88dc757f92d2aab" } new_table_name: "default.Table_da91e4435ef64c98bc07306bcb50023f" modify_external_catalogs: true
W20260430 19:58:46.264335 4161 consensus_peers.cc:597] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe -> Peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Couldn't send request to peer d8345ec7f57c4399a8a4afe4dd43439e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504523705ff944c7b3f84db0ddf9f455. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:46.265069 4750 consensus_queue.cc:237] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.267495 4803 tablet.cc:1724] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260430 19:58:46.287467 4317 consensus_peers.cc:597] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0f228d11b9641589825bb6a61310672. This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:46.287648 4015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260430 19:58:46.291124 4801 tablet.cc:1719] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Skipping requested alter to schema version 3, tablet already version 3
I20260430 19:58:46.292878 4744 raft_consensus.cc:493] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:46.293363 4744 raft_consensus.cc:515] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.295231 4744 leader_election.cc:290] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:46.299914 4742 tablet_replica.cc:333] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:46.306769 4029 master_service.cc:954] ReplaceTablet: received request to replace tablet d0f228d11b9641589825bb6a61310672 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:46.310495 4024 master_service.cc:954] ReplaceTablet: received request to replace tablet 41df32f61a7f412eac7059f589388dae from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:46.312398 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41df32f61a7f412eac7059f589388dae" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:46.313015 4531 raft_consensus.cc:2468] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:46.304556 4742 raft_consensus.cc:2243] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:46.314306 4317 leader_election.cc:304] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:46.315683 4742 raft_consensus.cc:2272] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:46.315917 4744 raft_consensus.cc:2804] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:46.316251 4744 raft_consensus.cc:493] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:46.316550 4744 raft_consensus.cc:3060] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.330478 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_58d8a52ec0924edcbe0bef79362e12c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.332212 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58d8a52ec0924edcbe0bef79362e12c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.337306 4020 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_1921e3a044f04ea8a445515ab59a8830" } modify_external_catalogs: true
I20260430 19:58:46.337930 4020 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_1921e3a044f04ea8a445515ab59a8830" } modify_external_catalogs: true
I20260430 19:58:46.341439 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.343065 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.349803 4744 raft_consensus.cc:515] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.351821 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba (DEFAULT_TABLE table=default.Table_ccf1123d6d5d4e489838f28c6125aa15 [id=007139834a09457dbfcb9a426839d894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.352773 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.355741 4802 tablet.cc:1719] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Skipping requested alter to schema version 3, tablet already version 3
W20260430 19:58:46.331077 4158 consensus_peers.cc:597] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504523705ff944c7b3f84db0ddf9f455. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:46.334455 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41df32f61a7f412eac7059f589388dae" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:46.363178 4750 raft_consensus.cc:493] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:46.334878 4803 tablet.cc:1719] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Skipping requested alter to schema version 3, tablet already version 3
I20260430 19:58:46.363613 4750 raft_consensus.cc:515] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.366469 4750 leader_election.cc:290] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:46.366676 4201 tablet_service.cc:1511] Processing CreateTablet for tablet d8e845c461084e33802aafa9c4d91e3e (DEFAULT_TABLE table=default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.367616 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76073435ab7949a0bb0606e6efaa48da" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:46.368459 4532 raft_consensus.cc:2393] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a7356e99e744525861e12c108cfa3fe in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:46.368422 4742 ts_tablet_manager.cc:1916] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.369863 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e845c461084e33802aafa9c4d91e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.372185 4776 tablet_bootstrap.cc:492] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:46.377835 4776 ts_tablet_manager.cc:1403] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.447s user 0.054s sys 0.116s
I20260430 19:58:46.377830 4512 tablet_service.cc:1511] Processing CreateTablet for tablet d8e845c461084e33802aafa9c4d91e3e (DEFAULT_TABLE table=default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.378502 4729 tablet_bootstrap.cc:492] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:46.379489 4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e845c461084e33802aafa9c4d91e3e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:46.384207 4016 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 404f20adf9d741f3a036b06cbdbaedb8 (table default.Table_936e51d7cb7f45d08b267025523f49a3 [id=06859fba4a5d4b8f978e6f56edc5fb85], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 404f20adf9d741f3a036b06cbdbaedb8 has no consensus state
I20260430 19:58:46.388185 4729 tablet_bootstrap.cc:654] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:46.393883 4776 raft_consensus.cc:359] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.399931 4511 tablet_service.cc:1511] Processing CreateTablet for tablet 9d1b860c7073446597ce696f822ce1c2 (DEFAULT_TABLE table=default.Table_b85a96ef82324def96137ff90837adf1 [id=3a1eab4e333c4999b8b5155702ad0e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.401109 4511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d1b860c7073446597ce696f822ce1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.402525 4776 raft_consensus.cc:385] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:46.403520 4776 raft_consensus.cc:740] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:46.405041 4776 consensus_queue.cc:260] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.399369 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 9d1b860c7073446597ce696f822ce1c2 (DEFAULT_TABLE table=default.Table_b85a96ef82324def96137ff90837adf1 [id=3a1eab4e333c4999b8b5155702ad0e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.410347 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d1b860c7073446597ce696f822ce1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.413038 4776 ts_tablet_manager.cc:1434] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.033s user 0.006s sys 0.001s
I20260430 19:58:46.419991 4742 ts_tablet_manager.cc:1929] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:46.420547 4742 log.cc:1199] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/238e99c55dd6480da589cf3f43483773
I20260430 19:58:46.422399 4744 leader_election.cc:290] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:46.422732 4742 ts_tablet_manager.cc:1950] T 238e99c55dd6480da589cf3f43483773 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:46.425137 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 238e99c55dd6480da589cf3f43483773 (table default.Table_0dcf8ddbe6eb490b819da1d1cbc96508 [id=b43a4d5da8c84baa89c5eefa1fbd8409]) successfully deleted
I20260430 19:58:46.413985 4776 tablet_bootstrap.cc:492] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:46.428370 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41df32f61a7f412eac7059f589388dae" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:46.428948 4532 raft_consensus.cc:3060] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.431200 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76073435ab7949a0bb0606e6efaa48da" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:46.427304 4320 leader_election.cc:343] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:46.432016 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_4bb2baa160334221824783b1ffff0cb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.433805 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4bb2baa160334221824783b1ffff0cb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.434267 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41df32f61a7f412eac7059f589388dae" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:46.434903 4222 raft_consensus.cc:3060] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.436167 4161 leader_election.cc:304] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:46.437219 4750 raft_consensus.cc:3060] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:46.444861 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 9d1b860c7073446597ce696f822ce1c2 (DEFAULT_TABLE table=default.Table_b85a96ef82324def96137ff90837adf1 [id=3a1eab4e333c4999b8b5155702ad0e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.445204 4363 tablet_service.cc:1511] Processing CreateTablet for tablet d8e845c461084e33802aafa9c4d91e3e (DEFAULT_TABLE table=default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.445955 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d1b860c7073446597ce696f822ce1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.445869 4750 raft_consensus.cc:2749] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:46.446442 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e845c461084e33802aafa9c4d91e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.453244 4531 raft_consensus.cc:1275] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Refusing update from remote peer d8345ec7f57c4399a8a4afe4dd43439e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:46.454706 4749 consensus_queue.cc:1048] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:58:46.459793 4026 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa59394fc34f419ca22a695b8dff4613 consensus state has no leader
I20260430 19:58:46.461741 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 43d96039a1d64306b04ee49a6b86e854
I20260430 19:58:46.465637 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d96039a1d64306b04ee49a6b86e854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.465862 4223 raft_consensus.cc:1275] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Refusing update from remote peer d8345ec7f57c4399a8a4afe4dd43439e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:46.466763 4737 tablet_replica.cc:333] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:46.467402 4737 raft_consensus.cc:2243] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:46.467888 4737 raft_consensus.cc:2272] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:46.471568 4776 tablet_bootstrap.cc:654] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:46.472623 4737 ts_tablet_manager.cc:1916] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.475711 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d96039a1d64306b04ee49a6b86e854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.476744 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_d38bd1441c3c4a89acfac535b88279a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:46.476421 4749 consensus_queue.cc:1048] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260430 19:58:46.478487 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d38bd1441c3c4a89acfac535b88279a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.483326 4742 tablet_replica.cc:333] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:46.485044 4742 raft_consensus.cc:2243] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:46.494455 4742 raft_consensus.cc:2272] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:46.495836 4532 raft_consensus.cc:2468] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:46.497148 4222 raft_consensus.cc:2468] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:46.491233 4776 tablet_bootstrap.cc:492] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:46.483855 4201 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d96039a1d64306b04ee49a6b86e854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.498427 4320 leader_election.cc:304] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:46.499240 4776 ts_tablet_manager.cc:1403] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.085s user 0.010s sys 0.001s
I20260430 19:58:46.499257 4749 raft_consensus.cc:2804] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:46.501220 4749 raft_consensus.cc:697] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:46.502033 4749 consensus_queue.cc:237] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.506166 4654 ts_tablet_manager.cc:1916] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.508219 4028 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 2 got 0
I20260430 19:58:46.516924 4742 ts_tablet_manager.cc:1916] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.533018 4024 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 41df32f61a7f412eac7059f589388dae
I20260430 19:58:46.534536 4024 catalog_manager.cc:6741] ReplaceTablet: tablet 41df32f61a7f412eac7059f589388dae deleted and replaced by tablet 4764dc427b4a48309de914c4f5e655fe
I20260430 19:58:46.534576 4201 tablet_service.cc:1558] Processing DeleteTablet for tablet 41df32f61a7f412eac7059f589388dae with delete_type TABLET_DATA_DELETED (replaced by tablet 4764dc427b4a48309de914c4f5e655fe) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.536593 4729 tablet_bootstrap.cc:492] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:46.537021 4729 ts_tablet_manager.cc:1403] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.159s user 0.026s sys 0.003s
I20260430 19:58:46.538228 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 41df32f61a7f412eac7059f589388dae with delete_type TABLET_DATA_DELETED (replaced by tablet 4764dc427b4a48309de914c4f5e655fe) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.539340 4729 raft_consensus.cc:359] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.540032 4729 raft_consensus.cc:385] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:46.540352 4729 raft_consensus.cc:740] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:46.541159 4729 consensus_queue.cc:260] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.544487 4028 catalog_manager.cc:5671] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a7356e99e744525861e12c108cfa3fe (127.3.190.1). New cstate: current_term: 1 leader_uuid: "5a7356e99e744525861e12c108cfa3fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:46.549265 4020 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 92447f04593d4323a3b44c3001b9bf9c
I20260430 19:58:46.549888 4513 tablet_service.cc:1511] Processing CreateTablet for tablet e36490392a404b6bba29bd6803274b36 (DEFAULT_TABLE table=default.Table_d904dde10c354ae5ab6b325322b1c0ae [id=cd706c6a34ac49a4b82e411aca27698a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.551340 4729 ts_tablet_manager.cc:1434] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260430 19:58:46.552803 4202 tablet_service.cc:1511] Processing CreateTablet for tablet e36490392a404b6bba29bd6803274b36 (DEFAULT_TABLE table=default.Table_d904dde10c354ae5ab6b325322b1c0ae [id=cd706c6a34ac49a4b82e411aca27698a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.553910 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36490392a404b6bba29bd6803274b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.502223 4776 raft_consensus.cc:359] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.561205 4776 raft_consensus.cc:385] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:46.562690 4776 raft_consensus.cc:740] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:46.564055 4818 tablet_bootstrap.cc:492] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:46.564791 4776 consensus_queue.cc:260] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.574618 4020 master_service.cc:954] ReplaceTablet: received request to replace tablet 3946fffc0d2f47f789c7341dc66d603e from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:46.579420 4514 tablet_service.cc:1511] Processing CreateTablet for tablet a4b0d830f1b14ce382b2e0242f6565e7 (DEFAULT_TABLE table=default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.576985 4203 tablet_service.cc:1511] Processing CreateTablet for tablet a4b0d830f1b14ce382b2e0242f6565e7 (DEFAULT_TABLE table=default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.581918 4510 tablet_service.cc:1511] Processing CreateTablet for tablet cfb8e475e89f434f98482fee1b45dbca (DEFAULT_TABLE table=default.Table_46b80f7ea2fc4fb5aadb474e5471fbb3 [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.583318 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb8e475e89f434f98482fee1b45dbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.585683 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36490392a404b6bba29bd6803274b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.589010 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b0d830f1b14ce382b2e0242f6565e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.590824 4737 ts_tablet_manager.cc:1929] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:46.591163 4737 log.cc:1199] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/43d96039a1d64306b04ee49a6b86e854
I20260430 19:58:46.591881 4737 ts_tablet_manager.cc:1950] T 43d96039a1d64306b04ee49a6b86e854 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:46.596169 4818 tablet_bootstrap.cc:654] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:46.612087 4729 tablet_bootstrap.cc:492] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:46.614564 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_b8df0c78d8ee4cc29163092181f1e028 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]) successfully deleted
I20260430 19:58:46.623026 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_05824f167de14d4bbfc98363345b38ee" } new_table_name: "default.Table_871afe5ef54a402c9d42abd2f1888527" modify_external_catalogs: true
I20260430 19:58:46.627401 4737 tablet_replica.cc:333] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:46.628044 4737 raft_consensus.cc:2243] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:46.635560 4020 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 3946fffc0d2f47f789c7341dc66d603e
I20260430 19:58:46.636960 4020 catalog_manager.cc:6741] ReplaceTablet: tablet 3946fffc0d2f47f789c7341dc66d603e deleted and replaced by tablet d4e34f3b1bf84f72855cf87683c5e318
I20260430 19:58:46.640323 4205 tablet_service.cc:1511] Processing CreateTablet for tablet fa59394fc34f419ca22a695b8dff4613 (DEFAULT_TABLE table=default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.641284 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa59394fc34f419ca22a695b8dff4613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.641549 4776 ts_tablet_manager.cc:1434] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.142s user 0.006s sys 0.000s
I20260430 19:58:46.636772 4512 tablet_service.cc:1511] Processing CreateTablet for tablet fa59394fc34f419ca22a695b8dff4613 (DEFAULT_TABLE table=default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.647336 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet 3946fffc0d2f47f789c7341dc66d603e with delete_type TABLET_DATA_DELETED (replaced by tablet d4e34f3b1bf84f72855cf87683c5e318) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.648962 4508 tablet_service.cc:1511] Processing CreateTablet for tablet c78d50690da7428fadb848dadefec47c (DEFAULT_TABLE table=default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.652699 4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa59394fc34f419ca22a695b8dff4613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.653292 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78d50690da7428fadb848dadefec47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.642424 4776 tablet_bootstrap.cc:492] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:46.643283 4201 tablet_service.cc:1511] Processing CreateTablet for tablet cfb8e475e89f434f98482fee1b45dbca (DEFAULT_TABLE table=default.Table_46b80f7ea2fc4fb5aadb474e5471fbb3 [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.644164 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b0d830f1b14ce382b2e0242f6565e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.644557 4204 tablet_service.cc:1558] Processing DeleteTablet for tablet 3946fffc0d2f47f789c7341dc66d603e with delete_type TABLET_DATA_DELETED (replaced by tablet d4e34f3b1bf84f72855cf87683c5e318) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.660404 4020 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 2 got 0
I20260430 19:58:46.664113 4729 tablet_bootstrap.cc:654] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:46.664456 4737 raft_consensus.cc:2272] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:46.665200 4028 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 37d6c54ae89946158cad45e741c1c0f0 has no consensus state
I20260430 19:58:46.666260 4737 ts_tablet_manager.cc:1916] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.671285 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_e0abcc152ab14699b99ab62f4bf1b0f8" } modify_external_catalogs: true
I20260430 19:58:46.671985 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_e0abcc152ab14699b99ab62f4bf1b0f8" } modify_external_catalogs: true
I20260430 19:58:46.679004 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb8e475e89f434f98482fee1b45dbca. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:46.678354 4320 consensus_peers.cc:597] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:46.682060 4355 tablet_service.cc:1558] Processing DeleteTablet for tablet 3946fffc0d2f47f789c7341dc66d603e with delete_type TABLET_DATA_DELETED (replaced by tablet d4e34f3b1bf84f72855cf87683c5e318) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.685444 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_fea5239c878e444d9865930366db22aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:46.685977 4356 tablet_service.cc:1511] Processing CreateTablet for tablet fa59394fc34f419ca22a695b8dff4613 (DEFAULT_TABLE table=default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.687515 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa59394fc34f419ca22a695b8dff4613. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:46.687595 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fea5239c878e444d9865930366db22aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.669484 4020 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_b8cbfa7ae3844f47b3e6606218439a88" } new_table_name: "default.Table_12fc75157d86414bac1739e26d54d739" modify_external_catalogs: true
I20260430 19:58:46.695125 4776 tablet_bootstrap.cc:654] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:46.701189 4204 tablet_service.cc:1511] Processing CreateTablet for tablet c78d50690da7428fadb848dadefec47c (DEFAULT_TABLE table=default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.702241 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78d50690da7428fadb848dadefec47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.715018 4357 tablet_service.cc:1511] Processing CreateTablet for tablet e36490392a404b6bba29bd6803274b36 (DEFAULT_TABLE table=default.Table_d904dde10c354ae5ab6b325322b1c0ae [id=cd706c6a34ac49a4b82e411aca27698a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.716352 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36490392a404b6bba29bd6803274b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.716051 4355 tablet_service.cc:1511] Processing CreateTablet for tablet c78d50690da7428fadb848dadefec47c (DEFAULT_TABLE table=default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.719642 4355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c78d50690da7428fadb848dadefec47c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:46.725521 4020 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c81b11280f6d49e681537c88fb1cbf68 has no consensus state
I20260430 19:58:46.724363 4200 tablet_service.cc:1511] Processing CreateTablet for tablet a7b625afadd744dc86d7bfe1da77876d (DEFAULT_TABLE table=default.Table_e0abcc152ab14699b99ab62f4bf1b0f8 [id=a0f9ecd533e74e449ffe30b7e4f274b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.727154 4200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b625afadd744dc86d7bfe1da77876d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.736536 4354 tablet_service.cc:1511] Processing CreateTablet for tablet a7b625afadd744dc86d7bfe1da77876d (DEFAULT_TABLE table=default.Table_e0abcc152ab14699b99ab62f4bf1b0f8 [id=a0f9ecd533e74e449ffe30b7e4f274b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.737782 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b625afadd744dc86d7bfe1da77876d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.754472 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 41df32f61a7f412eac7059f589388dae with delete_type TABLET_DATA_DELETED (replaced by tablet 4764dc427b4a48309de914c4f5e655fe) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.756287 4020 master_service.cc:954] ReplaceTablet: received request to replace tablet d8e845c461084e33802aafa9c4d91e3e from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:46.758414 4025 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_7740c2a925e845839ee3fa21dc4b4553" } modify_external_catalogs: true
I20260430 19:58:46.758993 4025 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_7740c2a925e845839ee3fa21dc4b4553" } modify_external_catalogs: true
I20260430 19:58:46.761808 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_5f1d98aaf557415b844cdab290064d24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.763521 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f1d98aaf557415b844cdab290064d24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.765399 4508 tablet_service.cc:1511] Processing CreateTablet for tablet a7b625afadd744dc86d7bfe1da77876d (DEFAULT_TABLE table=default.Table_e0abcc152ab14699b99ab62f4bf1b0f8 [id=a0f9ecd533e74e449ffe30b7e4f274b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.767271 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7b625afadd744dc86d7bfe1da77876d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.767441 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_7e8d384dfa62442fbf585d952b02fe29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.769083 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e8d384dfa62442fbf585d952b02fe29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.771569 4362 tablet_service.cc:1511] Processing CreateTablet for tablet cfb8e475e89f434f98482fee1b45dbca (DEFAULT_TABLE table=default.Table_46b80f7ea2fc4fb5aadb474e5471fbb3 [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.772734 4362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb8e475e89f434f98482fee1b45dbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.776291 4364 tablet_service.cc:1511] Processing CreateTablet for tablet a4b0d830f1b14ce382b2e0242f6565e7 (DEFAULT_TABLE table=default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:46.777787 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b0d830f1b14ce382b2e0242f6565e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:46.786038 4811 tablet.cc:1724] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:46.787642 4654 ts_tablet_manager.cc:1929] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:46.788182 4654 log.cc:1199] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/238e99c55dd6480da589cf3f43483773
I20260430 19:58:46.789342 4654 ts_tablet_manager.cc:1950] T 238e99c55dd6480da589cf3f43483773 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:46.796258 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_445a32d80e864772b0a615ef426ccadb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.797741 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_445a32d80e864772b0a615ef426ccadb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.804409 4803 tablet.cc:1724] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:46.814956 4022 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 2 got 0
I20260430 19:58:46.819264 4801 tablet.cc:1724] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:46.827229 4026 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]). Expected version 2 got 0
I20260430 19:58:46.827852 4654 tablet_replica.cc:333] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:46.828608 4654 raft_consensus.cc:2243] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:46.838819 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 238e99c55dd6480da589cf3f43483773 (table default.Table_0dcf8ddbe6eb490b819da1d1cbc96508 [id=b43a4d5da8c84baa89c5eefa1fbd8409]) successfully deleted
W20260430 19:58:46.839459 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet e42a2e81e4a44cc29f352d8d55a03c4a because the tablet was not found. No further retry: Not found: Tablet not found: e42a2e81e4a44cc29f352d8d55a03c4a
W20260430 19:58:46.839996 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet a8b08893b8a5417392dcb7bc957c938b because the tablet was not found. No further retry: Not found: Tablet not found: a8b08893b8a5417392dcb7bc957c938b
I20260430 19:58:46.850119 4025 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet b442d25f3b9944e8967171a5c5ae5247
I20260430 19:58:46.852344 4654 pending_rounds.cc:70] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:46.852638 4654 pending_rounds.cc:77] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7280964101436018688 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "d0f228d11b9641589825bb6a61310672" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_5ea41a3ae89e4e45b01d9a69e437c571" dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" new_extra_config { } }
W20260430 19:58:46.854050 4654 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53688: Aborted: Op aborted
I20260430 19:58:46.856644 4776 tablet_bootstrap.cc:492] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:46.859078 4020 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet d8e845c461084e33802aafa9c4d91e3e
I20260430 19:58:46.860908 4020 catalog_manager.cc:6741] ReplaceTablet: tablet d8e845c461084e33802aafa9c4d91e3e deleted and replaced by tablet 654df09f602b4f96bcf5d13a514df55c
I20260430 19:58:46.861395 4511 tablet_service.cc:1558] Processing DeleteTablet for tablet d8e845c461084e33802aafa9c4d91e3e with delete_type TABLET_DATA_DELETED (replaced by tablet 654df09f602b4f96bcf5d13a514df55c) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.863873 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet d8e845c461084e33802aafa9c4d91e3e with delete_type TABLET_DATA_DELETED (replaced by tablet 654df09f602b4f96bcf5d13a514df55c) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.865716 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet a7b625afadd744dc86d7bfe1da77876d
I20260430 19:58:46.868494 4511 tablet_service.cc:1558] Processing DeleteTablet for tablet a7b625afadd744dc86d7bfe1da77876d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:46.861439 4201 tablet_service.cc:1558] Processing DeleteTablet for tablet d8e845c461084e33802aafa9c4d91e3e with delete_type TABLET_DATA_DELETED (replaced by tablet 654df09f602b4f96bcf5d13a514df55c) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.868628 4204 tablet_service.cc:1558] Processing DeleteTablet for tablet a7b625afadd744dc86d7bfe1da77876d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:46.870048 4776 ts_tablet_manager.cc:1403] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.228s user 0.004s sys 0.009s
I20260430 19:58:46.874933 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4bb2baa160334221824783b1ffff0cb0" } new_table_name: "default.Table_e0a6f83819644441aa6fbbeb7868eefa" modify_external_catalogs: true
I20260430 19:58:46.875489 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet a7b625afadd744dc86d7bfe1da77876d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:46.886930 4803 tablet.cc:1719] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Skipping requested alter to schema version 2, tablet already version 2
I20260430 19:58:46.891117 4654 raft_consensus.cc:2272] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:46.892968 4818 tablet_bootstrap.cc:492] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:46.896370 4818 ts_tablet_manager.cc:1403] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.333s user 0.013s sys 0.009s
I20260430 19:58:46.893931 4024 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d] alter complete (version 2)
I20260430 19:58:46.897382 4776 raft_consensus.cc:359] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.898142 4776 raft_consensus.cc:385] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:46.903028 4769 raft_consensus.cc:493] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:46.903545 4769 raft_consensus.cc:515] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.889483 4801 tablet.cc:1719] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Skipping requested alter to schema version 2, tablet already version 2
I20260430 19:58:46.905277 4769 leader_election.cc:290] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:46.909078 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:46.910523 4470 leader_election.cc:343] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:46.912292 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:46.913723 4470 leader_election.cc:343] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:46.914130 4470 leader_election.cc:304] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:46.915262 4769 raft_consensus.cc:2749] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 19:58:46.917410 4015 consensus_peers.cc:597] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 -> Peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Couldn't send request to peer 6b0a440dabed4814aaefe431944a7b5e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260430 19:58:46.912683 4317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 120642us
I20260430 19:58:46.923331 4024 master_service.cc:954] ReplaceTablet: received request to replace tablet 43d96039a1d64306b04ee49a6b86e854 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:46.924072 4811 tablet.cc:1719] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Skipping requested alter to schema version 2, tablet already version 2
I20260430 19:58:46.927944 4654 ts_tablet_manager.cc:1916] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:46.930959 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_38209ef2ff664e2482f5795be37d93f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:46.932646 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38209ef2ff664e2482f5795be37d93f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:46.932297 4818 raft_consensus.cc:359] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.933269 4024 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5f1d98aaf557415b844cdab290064d24" } new_table_name: "default.Table_5cf52bfafdd9498bb018e3187203e10f" modify_external_catalogs: true
I20260430 19:58:46.933539 4818 raft_consensus.cc:385] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:46.933799 4818 raft_consensus.cc:740] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:46.934908 4818 consensus_queue.cc:260] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:46.927513 4028 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0832f9191b7f4c3683f37cce8966b266 has no consensus state
I20260430 19:58:46.898424 4776 raft_consensus.cc:740] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:46.938235 4776 consensus_queue.cc:260] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:46.958343 4024 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad2ce1d43f214c8d9c1a1001baf53440 has no consensus state
I20260430 19:58:46.960513 4742 ts_tablet_manager.cc:1929] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:46.960908 4742 log.cc:1199] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/d0f228d11b9641589825bb6a61310672
I20260430 19:58:46.961632 4742 ts_tablet_manager.cc:1950] T d0f228d11b9641589825bb6a61310672 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:46.968894 4776 ts_tablet_manager.cc:1434] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.094s user 0.007s sys 0.000s
I20260430 19:58:46.971312 4818 ts_tablet_manager.cc:1434] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.069s user 0.002s sys 0.006s
I20260430 19:58:46.980172 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet d0f228d11b9641589825bb6a61310672 (table default.Table_5ea41a3ae89e4e45b01d9a69e437c571 [id=50df359f50de4b88a31deb7922615f33]) successfully deleted
W20260430 19:58:46.983613 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 6a7a82e60c974997b52a26650b131fc0 because the tablet was not found. No further retry: Not found: Tablet not found: 6a7a82e60c974997b52a26650b131fc0
W20260430 19:58:46.985949 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet d1d52994ff694886b7d4ff4224367082 because the tablet was not found. No further retry: Not found: Tablet not found: d1d52994ff694886b7d4ff4224367082
I20260430 19:58:46.987752 4742 tablet_replica.cc:333] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:46.988214 4744 raft_consensus.cc:493] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:46.988700 4744 raft_consensus.cc:515] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:46.993707 4744 leader_election.cc:290] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:47.008849 4737 ts_tablet_manager.cc:1929] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:47.009238 4737 log.cc:1199] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/41df32f61a7f412eac7059f589388dae
I20260430 19:58:47.009824 4737 ts_tablet_manager.cc:1950] T 41df32f61a7f412eac7059f589388dae P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:47.011142 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:47.011698 4531 raft_consensus.cc:2468] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:47.012894 4317 leader_election.cc:304] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:47.013787 4744 raft_consensus.cc:2804] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:47.014099 4744 raft_consensus.cc:493] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:47.014470 4744 raft_consensus.cc:3060] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.015882 4818 tablet_bootstrap.cc:492] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:47.017055 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 41df32f61a7f412eac7059f589388dae (table default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]) successfully deleted
W20260430 19:58:47.018983 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 3946fffc0d2f47f789c7341dc66d603e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3946fffc0d2f47f789c7341dc66d603e already in progress: creating tablet
W20260430 19:58:47.020561 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet d8e845c461084e33802aafa9c4d91e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8e845c461084e33802aafa9c4d91e3e already in progress: creating tablet
I20260430 19:58:47.021929 4729 tablet_bootstrap.cc:492] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:47.022321 4729 ts_tablet_manager.cc:1403] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.410s user 0.006s sys 0.013s
W20260430 19:58:47.022895 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet a7b625afadd744dc86d7bfe1da77876d because the tablet was not found. No further retry: Not found: Tablet not found: a7b625afadd744dc86d7bfe1da77876d
I20260430 19:58:46.992005 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:47.027199 4320 leader_election.cc:343] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.028949 4729 raft_consensus.cc:359] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.029542 4729 raft_consensus.cc:385] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.029780 4729 raft_consensus.cc:740] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.006871 4776 tablet_bootstrap.cc:492] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:46.988505 4742 raft_consensus.cc:2243] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:47.032997 4742 pending_rounds.cc:70] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Trying to abort 1 pending ops.
I20260430 19:58:47.033314 4742 pending_rounds.cc:77] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964096599801856 op_type: NO_OP noop_request { }
I20260430 19:58:47.033772 4742 raft_consensus.cc:2889] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:47.045814 4750 consensus_queue.cc:1048] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:58:47.034192 4742 raft_consensus.cc:2272] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.062636 4729 consensus_queue.cc:260] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.064602 4729 ts_tablet_manager.cc:1434] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20260430 19:58:47.065490 4729 tablet_bootstrap.cc:492] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:47.068507 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_0a03c960230b4206944b333b146c39c7" } modify_external_catalogs: true
I20260430 19:58:47.069034 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_0a03c960230b4206944b333b146c39c7" } modify_external_catalogs: true
I20260430 19:58:47.073639 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a7e1ecd9b8b44627a65e244546a87c58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:47.075696 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_74eca774c5aa4af9ad279c22a8cff58e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:47.077073 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74eca774c5aa4af9ad279c22a8cff58e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.079063 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_28e920186f324cd09f7483671073fc82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:47.080634 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28e920186f324cd09f7483671073fc82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:47.084045 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7e1ecd9b8b44627a65e244546a87c58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.084432 4024 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ccf1123d6d5d4e489838f28c6125aa15" } modify_external_catalogs: true
I20260430 19:58:47.084674 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet 41df32f61a7f412eac7059f589388dae from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:47.084960 4024 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ccf1123d6d5d4e489838f28c6125aa15" } modify_external_catalogs: true
W20260430 19:58:47.089125 4320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 61989.8us
I20260430 19:58:47.082386 4742 ts_tablet_manager.cc:1916] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.097177 4508 tablet_service.cc:1558] Processing DeleteTablet for tablet 3946fffc0d2f47f789c7341dc66d603e with delete_type TABLET_DATA_DELETED (replaced by tablet d4e34f3b1bf84f72855cf87683c5e318) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:47.099264 4737 tablet_replica.cc:333] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:47.099943 4737 raft_consensus.cc:2243] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.100359 4737 raft_consensus.cc:2272] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.100179 4744 raft_consensus.cc:515] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.101672 4776 tablet_bootstrap.cc:654] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.102372 4737 ts_tablet_manager.cc:1916] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.138790 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_28e920186f324cd09f7483671073fc82" } new_table_name: "default.Table_f2973021c60246e299b6141f0e96e0e1" modify_external_catalogs: true
I20260430 19:58:47.142141 4742 ts_tablet_manager.cc:1929] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:47.142541 4742 log.cc:1199] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/504523705ff944c7b3f84db0ddf9f455
I20260430 19:58:47.143278 4742 ts_tablet_manager.cc:1950] T 504523705ff944c7b3f84db0ddf9f455 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:47.144063 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 38) took 1096 ms. Trace:
I20260430 19:58:47.144423 4742 rpcz_store.cc:276] 0430 19:58:46.047531 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.048293 (+ 762us) service_pool.cc:225] Handling call
0430 19:58:46.987601 (+939308us) ts_tablet_manager.cc:1151] Deleting tablet 504523705ff944c7b3f84db0ddf9f455
0430 19:58:46.987639 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.116337 (+128698us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.142007 (+ 25670us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.144003 (+ 1996us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":938799}
I20260430 19:58:47.145888 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 504523705ff944c7b3f84db0ddf9f455 (table default.Table_c884b9b4854b45a7ad0fd6e00fc8f21e [id=0f3d9fa7646043459be13fcce244dc0f]) successfully deleted
I20260430 19:58:47.146025 4750 raft_consensus.cc:493] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:47.146554 4750 raft_consensus.cc:515] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.148326 4776 tablet_bootstrap.cc:492] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:47.148705 4776 ts_tablet_manager.cc:1403] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.149s user 0.008s sys 0.005s
I20260430 19:58:47.150318 4750 leader_election.cc:290] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:47.150942 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e845c461084e33802aafa9c4d91e3e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:47.153043 4776 raft_consensus.cc:359] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.153746 4776 raft_consensus.cc:385] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.154016 4776 raft_consensus.cc:740] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
W20260430 19:58:47.156292 4158 leader_election.cc:343] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.156476 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_16cd323d5c83450f8e84d0eb40d80d10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:47.157015 4829 consensus_queue.cc:1048] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:58:47.157919 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16cd323d5c83450f8e84d0eb40d80d10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.154737 4776 consensus_queue.cc:260] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.163969 4020 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_936e51d7cb7f45d08b267025523f49a3" } new_table_name: "default.Table_2afe6c2b4071491f868a3a17cc4d6e48" modify_external_catalogs: true
I20260430 19:58:47.169423 4776 ts_tablet_manager.cc:1434] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.020s user 0.005s sys 0.003s
I20260430 19:58:47.174316 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet cfb8e475e89f434f98482fee1b45dbca from {username='slave'} at 127.0.0.1:55130
W20260430 19:58:47.174659 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet db0efaecfef44457a5128e878796c061 has no consensus state
I20260430 19:58:47.178332 4776 tablet_bootstrap.cc:492] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:47.188005 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 39) took 1140 ms. Trace:
I20260430 19:58:47.188403 4742 rpcz_store.cc:276] 0430 19:58:46.047841 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.053876 (+ 6035us) service_pool.cc:225] Handling call
0430 19:58:47.187774 (+1133898us) ts_tablet_manager.cc:1151] Deleting tablet a8b08893b8a5417392dcb7bc957c938b
0430 19:58:47.187814 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.187953 (+ 139us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1133315}
I20260430 19:58:47.189510 4742 tablet_replica.cc:333] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:47.190939 4742 raft_consensus.cc:2243] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.191448 4742 raft_consensus.cc:2272] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.193149 4742 ts_tablet_manager.cc:1916] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:47.191929 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet a8b08893b8a5417392dcb7bc957c938b because the tablet was not found. No further retry: Not found: Tablet not found: a8b08893b8a5417392dcb7bc957c938b
I20260430 19:58:47.217047 4776 tablet_bootstrap.cc:654] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.219274 4742 ts_tablet_manager.cc:1929] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:47.221575 4742 log.cc:1199] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/6108085995074ea890cc6933bf45661f
I20260430 19:58:47.222329 4742 ts_tablet_manager.cc:1950] T 6108085995074ea890cc6933bf45661f P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
W20260430 19:58:47.223052 4020 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 404f20adf9d741f3a036b06cbdbaedb8 has no consensus state
I20260430 19:58:47.223702 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 37) took 1176 ms. Trace:
I20260430 19:58:47.225615 4742 rpcz_store.cc:276] 0430 19:58:46.047199 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.054646 (+ 7447us) service_pool.cc:225] Handling call
0430 19:58:47.189368 (+1134722us) ts_tablet_manager.cc:1151] Deleting tablet 6108085995074ea890cc6933bf45661f
0430 19:58:47.189403 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.213187 (+ 23784us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.219159 (+ 5972us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.223649 (+ 4490us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1134221}
I20260430 19:58:47.110185 4654 ts_tablet_manager.cc:1929] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260430 19:58:47.227352 4654 log.cc:1199] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/d0f228d11b9641589825bb6a61310672
I20260430 19:58:47.228125 4654 ts_tablet_manager.cc:1950] T d0f228d11b9641589825bb6a61310672 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:47.114233 4818 tablet_bootstrap.cc:654] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.230680 4744 leader_election.cc:290] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:47.231515 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:47.232025 4531 raft_consensus.cc:3060] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.233079 4737 ts_tablet_manager.cc:1929] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:47.233412 4737 log.cc:1199] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/3946fffc0d2f47f789c7341dc66d603e
I20260430 19:58:47.233973 4737 ts_tablet_manager.cc:1950] T 3946fffc0d2f47f789c7341dc66d603e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:47.149219 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e845c461084e33802aafa9c4d91e3e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:47.235857 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 6108085995074ea890cc6933bf45661f (table default.Table_78209fb594f841308f82995e31eff165 [id=f509dee5b07f496381f8a0edcbb50ced]) successfully deleted
I20260430 19:58:47.236714 4742 tablet_replica.cc:333] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:47.237365 4742 raft_consensus.cc:2243] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.237461 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:47.238462 4742 raft_consensus.cc:2272] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:47.238842 4320 leader_election.cc:343] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.239132 4531 raft_consensus.cc:2468] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
W20260430 19:58:47.239554 4161 leader_election.cc:343] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.239930 4161 leader_election.cc:304] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:47.240330 4317 leader_election.cc:304] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:47.240664 4742 ts_tablet_manager.cc:1916] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.230378 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 32) took 1181 ms. Trace:
I20260430 19:58:47.242776 4829 raft_consensus.cc:2749] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:47.240937 4654 rpcz_store.cc:276] 0430 19:58:46.048587 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.051138 (+ 2551us) service_pool.cc:225] Handling call
0430 19:58:46.827710 (+776572us) ts_tablet_manager.cc:1151] Deleting tablet d0f228d11b9641589825bb6a61310672
0430 19:58:46.827755 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.005921 (+178166us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.110035 (+104114us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.230292 (+120257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4282,"tablet-delete.queue_time_us":776121}
I20260430 19:58:47.244065 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet d0f228d11b9641589825bb6a61310672 (table default.Table_5ea41a3ae89e4e45b01d9a69e437c571 [id=50df359f50de4b88a31deb7922615f33]) successfully deleted
I20260430 19:58:47.245020 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 37) took 1195 ms. Trace:
I20260430 19:58:47.246708 4654 rpcz_store.cc:276] 0430 19:58:46.050053 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.051814 (+ 1761us) service_pool.cc:225] Handling call
0430 19:58:47.244744 (+1192930us) ts_tablet_manager.cc:1151] Deleting tablet 6a7a82e60c974997b52a26650b131fc0
0430 19:58:47.244794 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.244963 (+ 169us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1192276}
W20260430 19:58:47.247895 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 6a7a82e60c974997b52a26650b131fc0 because the tablet was not found. No further retry: Not found: Tablet not found: 6a7a82e60c974997b52a26650b131fc0
I20260430 19:58:47.249351 4744 raft_consensus.cc:2804] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:47.249501 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 36) took 1199 ms. Trace:
I20260430 19:58:47.249643 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 3946fffc0d2f47f789c7341dc66d603e (table default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]) successfully deleted
I20260430 19:58:47.249827 4654 rpcz_store.cc:276] 0430 19:58:46.049772 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.053112 (+ 3340us) service_pool.cc:225] Handling call
0430 19:58:47.249214 (+1196102us) ts_tablet_manager.cc:1151] Deleting tablet 504523705ff944c7b3f84db0ddf9f455
0430 19:58:47.249273 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.249453 (+ 180us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1195300}
I20260430 19:58:47.250653 4729 tablet_bootstrap.cc:654] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:47.250905 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 504523705ff944c7b3f84db0ddf9f455 because the tablet was not found. No further retry: Not found: Tablet not found: 504523705ff944c7b3f84db0ddf9f455
I20260430 19:58:47.252759 4742 ts_tablet_manager.cc:1929] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:47.253185 4654 tablet_replica.cc:333] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:47.253914 4654 raft_consensus.cc:2243] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:47.255043 4744 raft_consensus.cc:697] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:47.258862 4654 raft_consensus.cc:2272] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.261029 4742 log.cc:1199] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/e42a2e81e4a44cc29f352d8d55a03c4a
I20260430 19:58:47.261600 4818 tablet_bootstrap.cc:492] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:47.261993 4818 ts_tablet_manager.cc:1403] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.246s user 0.006s sys 0.008s
I20260430 19:58:47.261718 4742 ts_tablet_manager.cc:1950] T e42a2e81e4a44cc29f352d8d55a03c4a P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:47.263346 4654 ts_tablet_manager.cc:1916] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.264442 4818 raft_consensus.cc:359] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.265370 4818 raft_consensus.cc:385] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.265727 4818 raft_consensus.cc:740] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.266891 4818 consensus_queue.cc:260] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.269237 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 40) took 1177 ms. Trace:
I20260430 19:58:47.259927 4744 consensus_queue.cc:237] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.269644 4742 rpcz_store.cc:276] 0430 19:58:46.092199 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.092320 (+ 121us) service_pool.cc:225] Handling call
0430 19:58:47.236570 (+1144250us) ts_tablet_manager.cc:1151] Deleting tablet e42a2e81e4a44cc29f352d8d55a03c4a
0430 19:58:47.236610 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.247034 (+ 10424us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.252653 (+ 5619us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.269178 (+ 16525us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1143701}
I20260430 19:58:47.270895 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet e42a2e81e4a44cc29f352d8d55a03c4a (table default.Table_ec013029c0de4691aee33eec63af4b84 [id=ca2b9e6cc2c646c0983b83946e91aa4f]) successfully deleted
I20260430 19:58:47.271720 4742 tablet_replica.cc:333] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:47.271759 4776 tablet_bootstrap.cc:492] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:47.272233 4776 ts_tablet_manager.cc:1403] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.094s user 0.012s sys 0.000s
I20260430 19:58:47.272543 4742 raft_consensus.cc:2243] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.273066 4742 raft_consensus.cc:2272] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.274457 4776 raft_consensus.cc:359] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.275022 4818 ts_tablet_manager.cc:1434] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.013s user 0.005s sys 0.003s
I20260430 19:58:47.275059 4776 raft_consensus.cc:385] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.275355 4776 raft_consensus.cc:740] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.278124 4742 ts_tablet_manager.cc:1916] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.278829 4776 consensus_queue.cc:260] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.283188 4818 tablet_bootstrap.cc:492] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:47.290483 4729 tablet_bootstrap.cc:492] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:47.290895 4729 ts_tablet_manager.cc:1403] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.226s user 0.009s sys 0.000s
I20260430 19:58:47.301947 4818 tablet_bootstrap.cc:654] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.286398 4776 ts_tablet_manager.cc:1434] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260430 19:58:47.303359 4776 tablet_bootstrap.cc:492] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:47.313100 4829 raft_consensus.cc:493] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:47.313551 4829 raft_consensus.cc:515] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.314029 4742 ts_tablet_manager.cc:1929] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:47.314342 4742 log.cc:1199] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/43d96039a1d64306b04ee49a6b86e854
I20260430 19:58:47.315024 4742 ts_tablet_manager.cc:1950] T 43d96039a1d64306b04ee49a6b86e854 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:47.315404 4829 leader_election.cc:290] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:47.317400 4742 tablet_replica.cc:333] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:47.318120 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_f3909b4f461d45dc8a93109062c91905"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:47.319857 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3909b4f461d45dc8a93109062c91905 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.321074 4742 raft_consensus.cc:2243] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.321497 4742 raft_consensus.cc:2272] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.322012 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_b8df0c78d8ee4cc29163092181f1e028 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]) successfully deleted
I20260430 19:58:47.323122 4742 ts_tablet_manager.cc:1916] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.330390 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3946fffc0d2f47f789c7341dc66d603e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:47.331108 4382 raft_consensus.cc:2468] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
W20260430 19:58:47.332162 4158 leader_election.cc:343] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 3946fffc0d2f47f789c7341dc66d603e
I20260430 19:58:47.332878 4161 leader_election.cc:304] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
W20260430 19:58:47.334930 4836 log.cc:927] Time spent T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260430 19:58:47.335731 4027 master_service.cc:954] ReplaceTablet: received request to replace tablet 2af571a9ee484ae5944d787def240c6e from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:47.338243 4818 tablet_bootstrap.cc:492] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:47.338665 4818 ts_tablet_manager.cc:1403] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.060s user 0.009s sys 0.005s
I20260430 19:58:47.339078 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_89f3aa5fc8b44b34b6bc2fe920db3e5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:47.340847 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89f3aa5fc8b44b34b6bc2fe920db3e5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.341725 4818 raft_consensus.cc:359] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.342365 4818 raft_consensus.cc:385] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.342636 4818 raft_consensus.cc:740] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.343492 4818 consensus_queue.cc:260] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.348932 4829 raft_consensus.cc:2804] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:47.349385 4829 raft_consensus.cc:493] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:47.349767 4829 raft_consensus.cc:3060] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.353302 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_cb48b45e8d654862861b868ea8fd0368"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:47.354835 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb48b45e8d654862861b868ea8fd0368 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.358670 4818 ts_tablet_manager.cc:1434] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260430 19:58:47.360778 4729 raft_consensus.cc:359] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.361399 4729 raft_consensus.cc:385] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.361644 4729 raft_consensus.cc:740] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.362582 4742 ts_tablet_manager.cc:1929] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:47.363689 4818 tablet_bootstrap.cc:492] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:47.364058 4742 log.cc:1199] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/41df32f61a7f412eac7059f589388dae
I20260430 19:58:47.368258 4742 ts_tablet_manager.cc:1950] T 41df32f61a7f412eac7059f589388dae P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:47.371965 4829 raft_consensus.cc:515] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.374882 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3946fffc0d2f47f789c7341dc66d603e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:47.375433 4382 raft_consensus.cc:3060] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.376072 4818 tablet_bootstrap.cc:654] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.380026 4021 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_f2973021c60246e299b6141f0e96e0e1" } new_table_name: "default.Table_1aa12b95330d4ea98efaa37dea13b6e7" modify_external_catalogs: true
I20260430 19:58:47.388485 4508 tablet_service.cc:1511] Processing CreateTablet for tablet ad2ce1d43f214c8d9c1a1001baf53440 (DEFAULT_TABLE table=default.Table_5cf52bfafdd9498bb018e3187203e10f [id=043f0b9bc9274b9fbdbdd17fb0d6689c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.389606 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2ce1d43f214c8d9c1a1001baf53440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.391472 4364 tablet_service.cc:1511] Processing CreateTablet for tablet ad2ce1d43f214c8d9c1a1001baf53440 (DEFAULT_TABLE table=default.Table_5cf52bfafdd9498bb018e3187203e10f [id=043f0b9bc9274b9fbdbdd17fb0d6689c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.392627 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2ce1d43f214c8d9c1a1001baf53440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.393954 4514 tablet_service.cc:1511] Processing CreateTablet for tablet f3db0439f4f042a4a9cb38ebd3764ab5 (DEFAULT_TABLE table=default.Table_7e8d384dfa62442fbf585d952b02fe29 [id=977b7bf53cdf45158ea1fb1a4d2fdf05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.394976 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3db0439f4f042a4a9cb38ebd3764ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.400981 4200 tablet_service.cc:1511] Processing CreateTablet for tablet ad2ce1d43f214c8d9c1a1001baf53440 (DEFAULT_TABLE table=default.Table_5cf52bfafdd9498bb018e3187203e10f [id=043f0b9bc9274b9fbdbdd17fb0d6689c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.402024 4200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2ce1d43f214c8d9c1a1001baf53440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.401986 4510 tablet_service.cc:1511] Processing CreateTablet for tablet 13e8206ac4484abfa769276c83fd3333 (DEFAULT_TABLE table=default.Table_d38bd1441c3c4a89acfac535b88279a8 [id=25d3d568199f429b8cc145e854ee730b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.403178 4742 tablet_replica.cc:333] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:47.403271 4512 tablet_service.cc:1511] Processing CreateTablet for tablet 0832f9191b7f4c3683f37cce8966b266 (DEFAULT_TABLE table=default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.404455 4511 tablet_service.cc:1511] Processing CreateTablet for tablet 2c119dd9a5d944eabd588f62a76da98e (DEFAULT_TABLE table=default.Table_445a32d80e864772b0a615ef426ccadb [id=51ad748be7c14d32ac99d6f8f63f569e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.404907 4776 tablet_bootstrap.cc:654] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.405774 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e8206ac4484abfa769276c83fd3333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.387427 4729 consensus_queue.cc:260] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.407612 4027 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 2af571a9ee484ae5944d787def240c6e
I20260430 19:58:47.407806 4742 raft_consensus.cc:2243] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.408218 4742 raft_consensus.cc:2272] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.408876 4027 catalog_manager.cc:6741] ReplaceTablet: tablet 2af571a9ee484ae5944d787def240c6e deleted and replaced by tablet 081a060c44924bcda09ffc8fe88012b3
I20260430 19:58:47.411006 4769 raft_consensus.cc:493] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:47.411442 4769 raft_consensus.cc:515] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.415664 4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0832f9191b7f4c3683f37cce8966b266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.415831 4654 ts_tablet_manager.cc:1929] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:47.416196 4654 log.cc:1199] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/43d96039a1d64306b04ee49a6b86e854
I20260430 19:58:47.419302 4511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c119dd9a5d944eabd588f62a76da98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.420462 4382 raft_consensus.cc:2468] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:47.421360 4654 ts_tablet_manager.cc:1950] T 43d96039a1d64306b04ee49a6b86e854 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:47.424774 4509 tablet_service.cc:1511] Processing CreateTablet for tablet edb14746d8564bf8833e933a01771492 (DEFAULT_TABLE table=default.Table_952e6a513f694f3ea455f5264631e8b8 [id=3d08f7ebc240499ab3f05c554244cfb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.425925 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb14746d8564bf8833e933a01771492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.430181 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 43d96039a1d64306b04ee49a6b86e854 (table default.Table_b8df0c78d8ee4cc29163092181f1e028 [id=abddf7bc9a6c4a76aa323bf84a2cc21b]) successfully deleted
I20260430 19:58:47.430546 4356 tablet_service.cc:1511] Processing CreateTablet for tablet f3db0439f4f042a4a9cb38ebd3764ab5 (DEFAULT_TABLE table=default.Table_7e8d384dfa62442fbf585d952b02fe29 [id=977b7bf53cdf45158ea1fb1a4d2fdf05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.431664 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3db0439f4f042a4a9cb38ebd3764ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.435308 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 13e8206ac4484abfa769276c83fd3333 (DEFAULT_TABLE table=default.Table_d38bd1441c3c4a89acfac535b88279a8 [id=25d3d568199f429b8cc145e854ee730b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.435730 4769 leader_election.cc:290] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:47.436297 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e8206ac4484abfa769276c83fd3333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.436398 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e845c461084e33802aafa9c4d91e3e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:47.439114 4470 leader_election.cc:343] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.439301 4355 tablet_service.cc:1511] Processing CreateTablet for tablet 0832f9191b7f4c3683f37cce8966b266 (DEFAULT_TABLE table=default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.440272 4355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0832f9191b7f4c3683f37cce8966b266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.442822 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 41df32f61a7f412eac7059f589388dae (table default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]) successfully deleted
I20260430 19:58:47.443308 4354 tablet_service.cc:1511] Processing CreateTablet for tablet 2c119dd9a5d944eabd588f62a76da98e (DEFAULT_TABLE table=default.Table_445a32d80e864772b0a615ef426ccadb [id=51ad748be7c14d32ac99d6f8f63f569e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.444336 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c119dd9a5d944eabd588f62a76da98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.434242 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_9d21910e9cb14cc6a75a5432b143c915"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:47.440991 4023 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ce9a7d51fd524fa6a8930979d0c5b125" } new_table_name: "default.Table_733dfaaf74774e8db1bfa300e4e56ced" modify_external_catalogs: true
I20260430 19:58:47.447396 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet d8e845c461084e33802aafa9c4d91e3e with delete_type TABLET_DATA_DELETED (replaced by tablet 654df09f602b4f96bcf5d13a514df55c) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:47.448884 4513 tablet_service.cc:1511] Processing CreateTablet for tablet d4e34f3b1bf84f72855cf87683c5e318 (DEFAULT_TABLE table=default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.449856 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4e34f3b1bf84f72855cf87683c5e318. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:47.450146 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d21910e9cb14cc6a75a5432b143c915 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.453454 4362 tablet_service.cc:1511] Processing CreateTablet for tablet edb14746d8564bf8833e933a01771492 (DEFAULT_TABLE table=default.Table_952e6a513f694f3ea455f5264631e8b8 [id=3d08f7ebc240499ab3f05c554244cfb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.454487 4362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb14746d8564bf8833e933a01771492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.454732 4204 tablet_service.cc:1511] Processing CreateTablet for tablet f3db0439f4f042a4a9cb38ebd3764ab5 (DEFAULT_TABLE table=default.Table_7e8d384dfa62442fbf585d952b02fe29 [id=977b7bf53cdf45158ea1fb1a4d2fdf05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.455749 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3db0439f4f042a4a9cb38ebd3764ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.457047 4507 tablet_service.cc:1511] Processing CreateTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 (DEFAULT_TABLE table=default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.458012 4507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d6c54ae89946158cad45e741c1c0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.459646 4363 tablet_service.cc:1511] Processing CreateTablet for tablet d4e34f3b1bf84f72855cf87683c5e318 (DEFAULT_TABLE table=default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.459666 4201 tablet_service.cc:1511] Processing CreateTablet for tablet 13e8206ac4484abfa769276c83fd3333 (DEFAULT_TABLE table=default.Table_d38bd1441c3c4a89acfac535b88279a8 [id=25d3d568199f429b8cc145e854ee730b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.460601 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4e34f3b1bf84f72855cf87683c5e318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.460628 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e8206ac4484abfa769276c83fd3333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.463383 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet 2af571a9ee484ae5944d787def240c6e with delete_type TABLET_DATA_DELETED (replaced by tablet 081a060c44924bcda09ffc8fe88012b3) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:47.462949 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet d8e845c461084e33802aafa9c4d91e3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8e845c461084e33802aafa9c4d91e3e already in progress: creating tablet
I20260430 19:58:47.463649 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 0832f9191b7f4c3683f37cce8966b266 (DEFAULT_TABLE table=default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.464510 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0832f9191b7f4c3683f37cce8966b266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.464383 4161 leader_election.cc:304] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
W20260430 19:58:47.464986 4161 raft_consensus.cc:2686] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260430 19:58:47.466060 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e845c461084e33802aafa9c4d91e3e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:47.466614 4223 raft_consensus.cc:2468] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:47.467801 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 2c119dd9a5d944eabd588f62a76da98e (DEFAULT_TABLE table=default.Table_445a32d80e864772b0a615ef426ccadb [id=51ad748be7c14d32ac99d6f8f63f569e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.467578 4470 leader_election.cc:304] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:47.468811 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c119dd9a5d944eabd588f62a76da98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.470901 4750 raft_consensus.cc:493] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:47.471295 4750 raft_consensus.cc:515] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.472141 4769 raft_consensus.cc:2804] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:47.472467 4769 raft_consensus.cc:493] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:47.472755 4769 raft_consensus.cc:3060] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.472954 4750 leader_election.cc:290] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:47.472890 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 654df09f602b4f96bcf5d13a514df55c (DEFAULT_TABLE table=default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.474026 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 654df09f602b4f96bcf5d13a514df55c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.475340 4203 tablet_service.cc:1511] Processing CreateTablet for tablet edb14746d8564bf8833e933a01771492 (DEFAULT_TABLE table=default.Table_952e6a513f694f3ea455f5264631e8b8 [id=3d08f7ebc240499ab3f05c554244cfb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.476332 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb14746d8564bf8833e933a01771492. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:47.481170 4013 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet d1d52994ff694886b7d4ff4224367082 (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet d1d52994ff694886b7d4ff4224367082 consensus state has no leader
I20260430 19:58:47.491200 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d1b860c7073446597ce696f822ce1c2" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:47.491822 4383 raft_consensus.cc:2468] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:47.496698 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 (DEFAULT_TABLE table=default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.497862 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d6c54ae89946158cad45e741c1c0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.497977 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 654df09f602b4f96bcf5d13a514df55c (DEFAULT_TABLE table=default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.498538 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 (DEFAULT_TABLE table=default.Table_0a03c960230b4206944b333b146c39c7 [id=26c468440dce47839c5c48e7bc76c173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.499037 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 654df09f602b4f96bcf5d13a514df55c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.498888 4161 leader_election.cc:304] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:47.499707 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6220177b3e3e4d18bd8e20e6d9d96be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.499696 4750 raft_consensus.cc:2804] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:47.500012 4750 raft_consensus.cc:493] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:47.500327 4750 raft_consensus.cc:3060] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.511451 4199 tablet_service.cc:1558] Processing DeleteTablet for tablet 2af571a9ee484ae5944d787def240c6e with delete_type TABLET_DATA_DELETED (replaced by tablet 081a060c44924bcda09ffc8fe88012b3) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:47.519515 4199 tablet_service.cc:1511] Processing CreateTablet for tablet d4e34f3b1bf84f72855cf87683c5e318 (DEFAULT_TABLE table=default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.519713 4769 raft_consensus.cc:515] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.520691 4654 tablet_replica.cc:333] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:47.521345 4654 raft_consensus.cc:2243] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.521756 4654 raft_consensus.cc:2272] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.525802 4199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4e34f3b1bf84f72855cf87683c5e318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.530483 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 (DEFAULT_TABLE table=default.Table_0a03c960230b4206944b333b146c39c7 [id=26c468440dce47839c5c48e7bc76c173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.531510 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6220177b3e3e4d18bd8e20e6d9d96be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.534279 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (DEFAULT_TABLE table=default.Table_fea5239c878e444d9865930366db22aa [id=2c86f02b8cd14685913a92ee25eb0483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.535487 4654 ts_tablet_manager.cc:1916] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.536996 4504 tablet_service.cc:1511] Processing CreateTablet for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (DEFAULT_TABLE table=default.Table_fea5239c878e444d9865930366db22aa [id=2c86f02b8cd14685913a92ee25eb0483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.538386 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cef0197c00b4ab7a11e2e5123a4c277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.547719 4504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cef0197c00b4ab7a11e2e5123a4c277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.549431 4359 tablet_service.cc:1511] Processing CreateTablet for tablet 404f20adf9d741f3a036b06cbdbaedb8 (DEFAULT_TABLE table=default.Table_2afe6c2b4071491f868a3a17cc4d6e48 [id=06859fba4a5d4b8f978e6f56edc5fb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.550541 4359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404f20adf9d741f3a036b06cbdbaedb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:47.564429 4317 consensus_peers.cc:597] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41df32f61a7f412eac7059f589388dae. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:47.581043 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 6220177b3e3e4d18bd8e20e6d9d96be7
I20260430 19:58:47.582160 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet cfb8e475e89f434f98482fee1b45dbca
I20260430 19:58:47.584164 4025 catalog_manager.cc:6741] ReplaceTablet: tablet cfb8e475e89f434f98482fee1b45dbca deleted and replaced by tablet 3423379d1e3d4020a4241bf8917f1050
I20260430 19:58:47.586154 4831 raft_consensus.cc:493] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:47.587121 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 404f20adf9d741f3a036b06cbdbaedb8 (DEFAULT_TABLE table=default.Table_2afe6c2b4071491f868a3a17cc4d6e48 [id=06859fba4a5d4b8f978e6f56edc5fb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.588028 4024 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 0e64ef047d8c46e9ac8387feb0dbe9ba
I20260430 19:58:47.589064 4353 tablet_service.cc:1558] Processing DeleteTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:47.592854 4021 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet db0efaecfef44457a5128e878796c061 has no consensus state
I20260430 19:58:47.596704 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:47.597815 4508 tablet_service.cc:1558] Processing DeleteTablet for tablet 2af571a9ee484ae5944d787def240c6e with delete_type TABLET_DATA_DELETED (replaced by tablet 081a060c44924bcda09ffc8fe88012b3) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:47.598680 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet cfb8e475e89f434f98482fee1b45dbca with delete_type TABLET_DATA_DELETED (replaced by tablet 3423379d1e3d4020a4241bf8917f1050) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:47.600263 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404f20adf9d741f3a036b06cbdbaedb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:47.600301 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6220177b3e3e4d18bd8e20e6d9d96be7 already in progress: creating tablet
I20260430 19:58:47.600354 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:47.601601 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet cfb8e475e89f434f98482fee1b45dbca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfb8e475e89f434f98482fee1b45dbca already in progress: creating tablet
I20260430 19:58:47.601794 4737 tablet_replica.cc:333] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:47.601891 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_5a90ad7e91c44b4ea6f4240ebe7e7d65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:47.602814 4737 raft_consensus.cc:2243] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.603679 4737 raft_consensus.cc:2272] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:47.603931 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a90ad7e91c44b4ea6f4240ebe7e7d65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.605674 4737 ts_tablet_manager.cc:1916] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:47.610244 4023 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 081a060c44924bcda09ffc8fe88012b3 has no consensus state
I20260430 19:58:47.612988 4729 ts_tablet_manager.cc:1434] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.322s user 0.012s sys 0.005s
I20260430 19:58:47.612598 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 (DEFAULT_TABLE table=default.Table_276bd93958054b6b9ad30f314bbb487c [id=de548c1ed3d8424b9a2d3c451444462c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.616474 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09f272340cd942e9b2af9e38a2bdec82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.622179 4352 tablet_service.cc:1558] Processing DeleteTablet for tablet cfb8e475e89f434f98482fee1b45dbca with delete_type TABLET_DATA_DELETED (replaced by tablet 3423379d1e3d4020a4241bf8917f1050) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:47.623937 4818 tablet_bootstrap.cc:492] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:47.628674 4351 tablet_service.cc:1511] Processing CreateTablet for tablet eafc058f392a45de9d30b521ebde0950 (DEFAULT_TABLE table=default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e [id=8d1c3bb61831442596e2112cef9032a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.586576 4831 raft_consensus.cc:515] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.629786 4508 tablet_service.cc:1511] Processing CreateTablet for tablet c81b11280f6d49e681537c88fb1cbf68 (DEFAULT_TABLE table=default.Table_12fc75157d86414bac1739e26d54d739 [id=7b43b6a692214e0b89c73b9374682197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.630653 4831 leader_election.cc:290] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:47.624382 4818 ts_tablet_manager.cc:1403] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.263s user 0.020s sys 0.006s
I20260430 19:58:47.632347 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c81b11280f6d49e681537c88fb1cbf68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.582870 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e845c461084e33802aafa9c4d91e3e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:47.634114 4223 raft_consensus.cc:3060] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.639626 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 (DEFAULT_TABLE table=default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.640646 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37d6c54ae89946158cad45e741c1c0f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.641657 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_abcdd9deb61842beb234605de671f749"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:47.642207 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 745f04d04091443ca1c39949d005dd69 (DEFAULT_TABLE table=default.Table_58d8a52ec0924edcbe0bef79362e12c6 [id=81f401e1f89444f6803873e7d3c6a1b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.643174 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745f04d04091443ca1c39949d005dd69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.643811 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d1b860c7073446597ce696f822ce1c2" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:47.644850 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:47.647406 4654 ts_tablet_manager.cc:1929] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260430 19:58:47.647014 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_abcdd9deb61842beb234605de671f749 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.601966 4505 tablet_service.cc:1511] Processing CreateTablet for tablet eafc058f392a45de9d30b521ebde0950 (DEFAULT_TABLE table=default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e [id=8d1c3bb61831442596e2112cef9032a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.651657 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eafc058f392a45de9d30b521ebde0950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.651338 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:47.654500 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 (DEFAULT_TABLE table=default.Table_276bd93958054b6b9ad30f314bbb487c [id=de548c1ed3d8424b9a2d3c451444462c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.655004 4360 tablet_service.cc:1511] Processing CreateTablet for tablet c81b11280f6d49e681537c88fb1cbf68 (DEFAULT_TABLE table=default.Table_12fc75157d86414bac1739e26d54d739 [id=7b43b6a692214e0b89c73b9374682197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.655337 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet d8e845c461084e33802aafa9c4d91e3e with delete_type TABLET_DATA_DELETED (replaced by tablet 654df09f602b4f96bcf5d13a514df55c) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:47.656816 4654 log.cc:1199] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/3946fffc0d2f47f789c7341dc66d603e
I20260430 19:58:47.657128 4198 tablet_service.cc:1511] Processing CreateTablet for tablet 654df09f602b4f96bcf5d13a514df55c (DEFAULT_TABLE table=default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.660682 4352 tablet_service.cc:1511] Processing CreateTablet for tablet 745f04d04091443ca1c39949d005dd69 (DEFAULT_TABLE table=default.Table_58d8a52ec0924edcbe0bef79362e12c6 [id=81f401e1f89444f6803873e7d3c6a1b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.661154 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e845c461084e33802aafa9c4d91e3e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:47.601346 4769 leader_election.cc:290] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
W20260430 19:58:47.663337 4470 leader_election.cc:343] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.663836 4654 ts_tablet_manager.cc:1950] T 3946fffc0d2f47f789c7341dc66d603e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:47.510373 4750 raft_consensus.cc:515] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.665310 4729 tablet_bootstrap.cc:492] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:47.664460 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745f04d04091443ca1c39949d005dd69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.668618 4503 tablet_service.cc:1511] Processing CreateTablet for tablet e4e87fda401047cfa7070ae95b1701a9 (DEFAULT_TABLE table=default.Table_4b6fc137cf6945079fa8310db94a2b1a [id=2cb52e96af9f4eea998718eef0ac58ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.670079 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e87fda401047cfa7070ae95b1701a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.671005 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eafc058f392a45de9d30b521ebde0950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.668322 4197 tablet_service.cc:1511] Processing CreateTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 (DEFAULT_TABLE table=default.Table_0a03c960230b4206944b333b146c39c7 [id=26c468440dce47839c5c48e7bc76c173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.675252 4197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6220177b3e3e4d18bd8e20e6d9d96be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.676822 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e64ef047d8c46e9ac8387feb0dbe9ba" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:47.677515 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09f272340cd942e9b2af9e38a2bdec82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.684551 4223 raft_consensus.cc:2468] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:47.688086 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 53) took 1010 ms. Trace:
I20260430 19:58:47.688961 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c81b11280f6d49e681537c88fb1cbf68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.689826 4818 raft_consensus.cc:359] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.694392 4358 tablet_service.cc:1511] Processing CreateTablet for tablet e4e87fda401047cfa7070ae95b1701a9 (DEFAULT_TABLE table=default.Table_4b6fc137cf6945079fa8310db94a2b1a [id=2cb52e96af9f4eea998718eef0ac58ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.696018 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_0c33415d44d44344a5740dc6cdb7da0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:47.697778 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c33415d44d44344a5740dc6cdb7da0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:47.697947 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e87fda401047cfa7070ae95b1701a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.695346 4729 tablet_bootstrap.cc:654] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.707139 4818 raft_consensus.cc:385] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.707450 4818 raft_consensus.cc:740] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.708149 4818 consensus_queue.cc:260] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.714998 4509 tablet_service.cc:1511] Processing CreateTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 (DEFAULT_TABLE table=default.Table_95bf05b38f054143a0f537b967e00b93 [id=28b54770e4484fb28a9ceb33742ba620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.716173 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e62335e43c1410b8b3acdd9e2f47981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.695469 4654 rpcz_store.cc:276] 0430 19:58:46.677662 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.681644 (+ 3982us) service_pool.cc:225] Handling call
0430 19:58:47.520543 (+838899us) ts_tablet_manager.cc:1151] Deleting tablet 3946fffc0d2f47f789c7341dc66d603e
0430 19:58:47.520585 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.578740 (+ 58155us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.647305 (+ 68565us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.688028 (+ 40723us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":835975}
I20260430 19:58:47.718748 4654 tablet_replica.cc:333] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:47.719377 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 3946fffc0d2f47f789c7341dc66d603e (table default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]) successfully deleted
I20260430 19:58:47.723258 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 (DEFAULT_TABLE table=default.Table_95bf05b38f054143a0f537b967e00b93 [id=28b54770e4484fb28a9ceb33742ba620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.724390 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e62335e43c1410b8b3acdd9e2f47981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.727010 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d1b860c7073446597ce696f822ce1c2" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:47.727530 4383 raft_consensus.cc:3060] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:47.738734 4818 ts_tablet_manager.cc:1434] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.107s user 0.004s sys 0.006s
I20260430 19:58:47.738853 4383 raft_consensus.cc:2468] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:47.740293 4654 raft_consensus.cc:2243] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:47.741004 4654 pending_rounds.cc:70] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:47.741271 4654 pending_rounds.cc:77] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964102160105472 op_type: NO_OP noop_request { }
I20260430 19:58:47.741640 4654 raft_consensus.cc:2889] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:47.741982 4654 raft_consensus.cc:2272] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.743927 4654 ts_tablet_manager.cc:1916] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.744030 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d1b860c7073446597ce696f822ce1c2" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:47.745577 4818 tablet_bootstrap.cc:492] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:47.702865 4196 tablet_service.cc:1511] Processing CreateTablet for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (DEFAULT_TABLE table=default.Table_fea5239c878e444d9865930366db22aa [id=2c86f02b8cd14685913a92ee25eb0483]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.704349 4198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 654df09f602b4f96bcf5d13a514df55c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.764025 4737 ts_tablet_manager.cc:1929] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:47.718713 4195 tablet_service.cc:1558] Processing DeleteTablet for tablet cfb8e475e89f434f98482fee1b45dbca with delete_type TABLET_DATA_DELETED (replaced by tablet 3423379d1e3d4020a4241bf8917f1050) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:47.721508 4194 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e64ef047d8c46e9ac8387feb0dbe9ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:47.723572 4193 tablet_service.cc:1511] Processing CreateTablet for tablet eafc058f392a45de9d30b521ebde0950 (DEFAULT_TABLE table=default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e [id=8d1c3bb61831442596e2112cef9032a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.766129 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eafc058f392a45de9d30b521ebde0950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.771207 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cef0197c00b4ab7a11e2e5123a4c277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.776242 4737 log.cc:1199] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/0e64ef047d8c46e9ac8387feb0dbe9ba
I20260430 19:58:47.775417 4503 tablet_service.cc:1511] Processing CreateTablet for tablet 4764dc427b4a48309de914c4f5e655fe (DEFAULT_TABLE table=default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.778237 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4764dc427b4a48309de914c4f5e655fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.727077 4750 leader_election.cc:290] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:47.736022 4192 tablet_service.cc:1511] Processing CreateTablet for tablet c81b11280f6d49e681537c88fb1cbf68 (DEFAULT_TABLE table=default.Table_12fc75157d86414bac1739e26d54d739 [id=7b43b6a692214e0b89c73b9374682197]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.782135 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c81b11280f6d49e681537c88fb1cbf68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.738088 4191 tablet_service.cc:1511] Processing CreateTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 (DEFAULT_TABLE table=default.Table_276bd93958054b6b9ad30f314bbb487c [id=de548c1ed3d8424b9a2d3c451444462c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.786248 4191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09f272340cd942e9b2af9e38a2bdec82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.789194 4737 ts_tablet_manager.cc:1950] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:47.739836 4190 tablet_service.cc:1511] Processing CreateTablet for tablet 745f04d04091443ca1c39949d005dd69 (DEFAULT_TABLE table=default.Table_58d8a52ec0924edcbe0bef79362e12c6 [id=81f401e1f89444f6803873e7d3c6a1b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.790733 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745f04d04091443ca1c39949d005dd69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.791712 4729 tablet_bootstrap.cc:492] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:47.792114 4729 ts_tablet_manager.cc:1403] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.127s user 0.013s sys 0.000s
I20260430 19:58:47.793890 4737 tablet_replica.cc:333] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:47.740231 4161 leader_election.cc:304] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:47.794075 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 0e64ef047d8c46e9ac8387feb0dbe9ba (table default.Table_ccf1123d6d5d4e489838f28c6125aa15 [id=007139834a09457dbfcb9a426839d894]) successfully deleted
W20260430 19:58:47.794909 4161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 54774.7us
I20260430 19:58:47.741662 4189 tablet_service.cc:1511] Processing CreateTablet for tablet e4e87fda401047cfa7070ae95b1701a9 (DEFAULT_TABLE table=default.Table_4b6fc137cf6945079fa8310db94a2b1a [id=2cb52e96af9f4eea998718eef0ac58ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.796253 4189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e87fda401047cfa7070ae95b1701a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.797106 4818 tablet_bootstrap.cc:654] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.802565 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4b6fc137cf6945079fa8310db94a2b1a" } modify_external_catalogs: true
I20260430 19:58:47.803125 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4b6fc137cf6945079fa8310db94a2b1a" } modify_external_catalogs: true
I20260430 19:58:47.803857 4025 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_d38bd1441c3c4a89acfac535b88279a8" } modify_external_catalogs: true
I20260430 19:58:47.804311 4025 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_d38bd1441c3c4a89acfac535b88279a8" } modify_external_catalogs: true
I20260430 19:58:47.805531 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_46b80f7ea2fc4fb5aadb474e5471fbb3" } new_table_name: "default.Table_d13179c843c54969b98111d67f829f5f" modify_external_catalogs: true
I20260430 19:58:47.806666 4831 raft_consensus.cc:2804] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:47.807214 4737 raft_consensus.cc:2243] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.816361 4737 raft_consensus.cc:2272] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.820109 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_1aa12b95330d4ea98efaa37dea13b6e7" } new_table_name: "default.Table_b4bb392178ef4bdbbcc1375b8497375e" modify_external_catalogs: true
I20260430 19:58:47.808928 4729 raft_consensus.cc:359] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.825469 4729 raft_consensus.cc:385] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.825775 4729 raft_consensus.cc:740] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.826382 4729 consensus_queue.cc:260] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.828729 4729 ts_tablet_manager.cc:1434] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260430 19:58:47.829578 4729 tablet_bootstrap.cc:492] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:47.830901 4654 ts_tablet_manager.cc:1929] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:47.836071 4737 ts_tablet_manager.cc:1916] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.843869 4654 log.cc:1199] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/41df32f61a7f412eac7059f589388dae
I20260430 19:58:47.845077 4654 ts_tablet_manager.cc:1950] T 41df32f61a7f412eac7059f589388dae P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
W20260430 19:58:47.744132 4158 leader_election.cc:343] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.746847 4202 tablet_service.cc:1558] Processing DeleteTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:47.752344 4776 tablet_bootstrap.cc:492] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:47.850245 4776 ts_tablet_manager.cc:1403] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.547s user 0.019s sys 0.009s
I20260430 19:58:47.851743 4729 tablet_bootstrap.cc:654] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.856032 4776 raft_consensus.cc:359] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.856674 4776 raft_consensus.cc:385] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.856938 4776 raft_consensus.cc:740] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.857586 4776 consensus_queue.cc:260] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.859427 4776 ts_tablet_manager.cc:1434] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260430 19:58:47.860314 4776 tablet_bootstrap.cc:492] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:47.876310 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 404f20adf9d741f3a036b06cbdbaedb8 (DEFAULT_TABLE table=default.Table_2afe6c2b4071491f868a3a17cc4d6e48 [id=06859fba4a5d4b8f978e6f56edc5fb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.878718 4470 leader_election.cc:304] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:47.879829 4769 raft_consensus.cc:2804] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:47.880267 4769 raft_consensus.cc:697] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:47.881002 4769 consensus_queue.cc:237] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:47.882815 4158 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 138722us
W20260430 19:58:47.883494 4158 leader_election.cc:343] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:47.883870 4158 leader_election.cc:304] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
W20260430 19:58:47.884925 4158 leader_election.cc:343] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:47.886794 4750 raft_consensus.cc:2749] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:47.891407 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 48) took 1215 ms. Trace:
I20260430 19:58:47.891716 4654 rpcz_store.cc:276] 0430 19:58:46.676070 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.754132 (+ 78062us) service_pool.cc:225] Handling call
0430 19:58:47.718581 (+964449us) ts_tablet_manager.cc:1151] Deleting tablet 41df32f61a7f412eac7059f589388dae
0430 19:58:47.718646 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.779276 (+ 60630us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.830774 (+ 51498us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:47.891339 (+ 60565us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20822,"tablet-delete.queue_time_us":963563}
I20260430 19:58:47.892755 4654 tablet_replica.cc:333] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:47.893378 4654 raft_consensus.cc:2243] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:47.893765 4654 raft_consensus.cc:2272] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:47.899149 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 41df32f61a7f412eac7059f589388dae (table default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]) successfully deleted
I20260430 19:58:47.912523 4654 ts_tablet_manager.cc:1916] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:47.912467 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 4764dc427b4a48309de914c4f5e655fe (DEFAULT_TABLE table=default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:47.913713 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4764dc427b4a48309de914c4f5e655fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:47.932384 4818 tablet_bootstrap.cc:492] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:47.932830 4818 ts_tablet_manager.cc:1403] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.188s user 0.013s sys 0.007s
I20260430 19:58:47.935473 4818 raft_consensus.cc:359] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.936115 4818 raft_consensus.cc:385] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:47.936487 4818 raft_consensus.cc:740] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:47.937103 4818 consensus_queue.cc:260] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:47.963388 4818 ts_tablet_manager.cc:1434] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.030s user 0.006s sys 0.001s
I20260430 19:58:47.965972 4737 ts_tablet_manager.cc:1929] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:47.966363 4737 log.cc:1199] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/2af571a9ee484ae5944d787def240c6e
I20260430 19:58:47.979194 4776 tablet_bootstrap.cc:654] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:47.979101 4831 raft_consensus.cc:697] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:47.982663 4729 tablet_bootstrap.cc:492] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:47.983191 4729 ts_tablet_manager.cc:1403] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.154s user 0.045s sys 0.049s
I20260430 19:58:47.984809 4818 tablet_bootstrap.cc:492] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:47.985663 4729 raft_consensus.cc:359] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.011585 4729 raft_consensus.cc:385] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.012356 4729 raft_consensus.cc:740] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.013865 4729 consensus_queue.cc:260] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.016194 4204 tablet_service.cc:1511] Processing CreateTablet for tablet 4764dc427b4a48309de914c4f5e655fe (DEFAULT_TABLE table=default.Table_a2efa2a9f9bd42738350b731f2debc69 [id=4b4c0010ad354dd28be8cc7142d321c3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:48.017226 4028 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3423379d1e3d4020a4241bf8917f1050 has no consensus state
I20260430 19:58:48.019841 4201 tablet_service.cc:1511] Processing CreateTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 (DEFAULT_TABLE table=default.Table_95bf05b38f054143a0f537b967e00b93 [id=28b54770e4484fb28a9ceb33742ba620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.024567 4829 leader_election.cc:290] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:48.026450 4742 ts_tablet_manager.cc:1916] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.026777 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet e4e87fda401047cfa7070ae95b1701a9
I20260430 19:58:47.967065 4737 ts_tablet_manager.cc:1950] T 2af571a9ee484ae5944d787def240c6e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:48.028594 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet e4e87fda401047cfa7070ae95b1701a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:47.983525 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404f20adf9d741f3a036b06cbdbaedb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:48.032431 4158 leader_election.cc:343] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 3946fffc0d2f47f789c7341dc66d603e
I20260430 19:58:48.034823 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e62335e43c1410b8b3acdd9e2f47981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.034862 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet e4e87fda401047cfa7070ae95b1701a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:48.035003 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet e4e87fda401047cfa7070ae95b1701a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.051229 4729 ts_tablet_manager.cc:1434] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.068s user 0.004s sys 0.001s
I20260430 19:58:48.056536 4729 tablet_bootstrap.cc:492] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:48.064479 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4764dc427b4a48309de914c4f5e655fe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:48.069597 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet db0efaecfef44457a5128e878796c061 has no consensus state
I20260430 19:58:48.082329 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 2af571a9ee484ae5944d787def240c6e (table default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]) successfully deleted
I20260430 19:58:48.083349 4737 tablet_replica.cc:333] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:48.084101 4737 raft_consensus.cc:2243] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:48.086006 4729 tablet_bootstrap.cc:654] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.114769 4776 tablet_bootstrap.cc:492] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:48.115149 4729 tablet_bootstrap.cc:492] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:48.115279 4776 ts_tablet_manager.cc:1403] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.255s user 0.009s sys 0.006s
I20260430 19:58:48.115549 4729 ts_tablet_manager.cc:1403] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.002s
I20260430 19:58:48.111159 4831 consensus_queue.cc:237] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.118448 4729 raft_consensus.cc:359] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:48.119081 4729 raft_consensus.cc:385] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.119344 4729 raft_consensus.cc:740] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.119997 4729 consensus_queue.cc:260] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:48.130883 4509 tablet_service.cc:1511] Processing CreateTablet for tablet 798bb6007af849e99bf8e95d84624044 (DEFAULT_TABLE table=default.Table_89f3aa5fc8b44b34b6bc2fe920db3e5e [id=d963a034463343989c1d699cce2134a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.132053 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 798bb6007af849e99bf8e95d84624044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.136792 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 3946fffc0d2f47f789c7341dc66d603e with delete_type TABLET_DATA_DELETED (replaced by tablet d4e34f3b1bf84f72855cf87683c5e318) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.142933 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 798bb6007af849e99bf8e95d84624044 (DEFAULT_TABLE table=default.Table_89f3aa5fc8b44b34b6bc2fe920db3e5e [id=d963a034463343989c1d699cce2134a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.144142 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 798bb6007af849e99bf8e95d84624044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.153645 4503 tablet_service.cc:1511] Processing CreateTablet for tablet ed157967cdad480f88963a59fe85b54b (DEFAULT_TABLE table=default.Table_f3909b4f461d45dc8a93109062c91905 [id=64aae7c9cffe499eb723030d1a14697e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.155059 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed157967cdad480f88963a59fe85b54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.163177 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_308a505e52094e64933fe14eee977c7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:48.164806 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_308a505e52094e64933fe14eee977c7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.084818 4737 pending_rounds.cc:70] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Trying to abort 1 pending ops.
I20260430 19:58:48.165668 4737 pending_rounds.cc:77] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964107813007360 op_type: NO_OP noop_request { }
I20260430 19:58:48.165988 4737 raft_consensus.cc:2889] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:48.166287 4729 ts_tablet_manager.cc:1434] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.050s user 0.005s sys 0.000s
I20260430 19:58:48.166311 4737 raft_consensus.cc:2272] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.167546 4190 tablet_service.cc:1511] Processing CreateTablet for tablet 798bb6007af849e99bf8e95d84624044 (DEFAULT_TABLE table=default.Table_89f3aa5fc8b44b34b6bc2fe920db3e5e [id=d963a034463343989c1d699cce2134a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.168690 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 798bb6007af849e99bf8e95d84624044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.172451 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 13e8206ac4484abfa769276c83fd3333
I20260430 19:58:48.174397 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e8206ac4484abfa769276c83fd3333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.176004 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e8206ac4484abfa769276c83fd3333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:48.180295 4020 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:48.181630 4737 ts_tablet_manager.cc:1916] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.181887 4199 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e8206ac4484abfa769276c83fd3333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:48.209425 4357 tablet_service.cc:1511] Processing CreateTablet for tablet ed157967cdad480f88963a59fe85b54b (DEFAULT_TABLE table=default.Table_f3909b4f461d45dc8a93109062c91905 [id=64aae7c9cffe499eb723030d1a14697e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.210644 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed157967cdad480f88963a59fe85b54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.227505 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 67e5cb3aaa4044eb9dd06227882e2b42 (DEFAULT_TABLE table=default.Table_74eca774c5aa4af9ad279c22a8cff58e [id=654c558752824fcfb216a4b8b0c85ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.228593 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e5cb3aaa4044eb9dd06227882e2b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.209650 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 67e5cb3aaa4044eb9dd06227882e2b42 (DEFAULT_TABLE table=default.Table_74eca774c5aa4af9ad279c22a8cff58e [id=654c558752824fcfb216a4b8b0c85ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.233103 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e5cb3aaa4044eb9dd06227882e2b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.235913 4776 raft_consensus.cc:359] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.236570 4776 raft_consensus.cc:385] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.236876 4776 raft_consensus.cc:740] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.237490 4776 consensus_queue.cc:260] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.237599 4199 tablet_service.cc:1511] Processing CreateTablet for tablet ed157967cdad480f88963a59fe85b54b (DEFAULT_TABLE table=default.Table_f3909b4f461d45dc8a93109062c91905 [id=64aae7c9cffe499eb723030d1a14697e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.238768 4199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed157967cdad480f88963a59fe85b54b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:48.243454 4158 consensus_peers.cc:597] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2af571a9ee484ae5944d787def240c6e. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:48.248826 4742 ts_tablet_manager.cc:1929] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:48.249341 4742 log.cc:1199] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/3946fffc0d2f47f789c7341dc66d603e
I20260430 19:58:48.215755 4729 tablet_bootstrap.cc:492] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:48.215489 4508 tablet_service.cc:1511] Processing CreateTablet for tablet 51e35e0981e649f684caa96499040c15 (DEFAULT_TABLE table=default.Table_a7e1ecd9b8b44627a65e244546a87c58 [id=56ead7419d654e5dad1631c264ec87a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.250207 4742 ts_tablet_manager.cc:1950] T 3946fffc0d2f47f789c7341dc66d603e P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:48.250689 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e35e0981e649f684caa96499040c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.255107 4729 tablet_bootstrap.cc:654] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.261593 4201 tablet_service.cc:1511] Processing CreateTablet for tablet 67e5cb3aaa4044eb9dd06227882e2b42 (DEFAULT_TABLE table=default.Table_74eca774c5aa4af9ad279c22a8cff58e [id=654c558752824fcfb216a4b8b0c85ad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.263557 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e5cb3aaa4044eb9dd06227882e2b42. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:48.263931 4020 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2], current schema version=4): Not found: failed to reset TS proxy: No leader found: Tablet c78d50690da7428fadb848dadefec47c consensus state has no leader
I20260430 19:58:48.266053 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_c4c6ff375216499ba02c2f6fb6ad2db5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:48.266853 4654 ts_tablet_manager.cc:1929] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:48.267185 4654 log.cc:1199] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/d8e845c461084e33802aafa9c4d91e3e
I20260430 19:58:48.267771 4654 ts_tablet_manager.cc:1950] T d8e845c461084e33802aafa9c4d91e3e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
W20260430 19:58:48.267741 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4c6ff375216499ba02c2f6fb6ad2db5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.262622 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 50) took 1621 ms. Trace:
I20260430 19:58:48.270712 4742 rpcz_store.cc:276] 0430 19:58:46.641547 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.644435 (+ 2888us) service_pool.cc:225] Handling call
0430 19:58:47.403043 (+758608us) ts_tablet_manager.cc:1151] Deleting tablet 3946fffc0d2f47f789c7341dc66d603e
0430 19:58:47.403080 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.071896 (+668816us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.248709 (+176813us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.262532 (+ 13823us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":728615}
I20260430 19:58:48.271427 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cb48b45e8d654862861b868ea8fd0368" } modify_external_catalogs: true
I20260430 19:58:48.271996 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 3946fffc0d2f47f789c7341dc66d603e (table default.Table_8807d5c56dd74765bbf890fd151101ef [id=9713902a096f4953a4287b4078bea519]) successfully deleted
I20260430 19:58:48.271951 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cb48b45e8d654862861b868ea8fd0368" } modify_external_catalogs: true
I20260430 19:58:48.264591 4776 ts_tablet_manager.cc:1434] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.149s user 0.006s sys 0.002s
I20260430 19:58:48.272635 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_40c9009b608449f5ad959e33a2fbbc12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:48.273306 4776 tablet_bootstrap.cc:492] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
W20260430 19:58:48.274315 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40c9009b608449f5ad959e33a2fbbc12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.277525 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_74850f27abe54175909dc1d9c494804b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:48.279174 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74850f27abe54175909dc1d9c494804b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.281327 4742 tablet_replica.cc:333] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:48.282033 4742 raft_consensus.cc:2243] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:48.282439 4742 raft_consensus.cc:2272] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.284308 4020 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5cf52bfafdd9498bb018e3187203e10f" } new_table_name: "default.Table_cf6f9cf5be37436f9671fa67edc046a1" modify_external_catalogs: true
I20260430 19:58:48.286273 4742 ts_tablet_manager.cc:1916] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.286995 4029 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_8807d5c56dd74765bbf890fd151101ef" } new_table_name: "default.Table_e298a596df714d4494a339954849530b" modify_external_catalogs: true
I20260430 19:58:48.291528 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 51e35e0981e649f684caa96499040c15 (DEFAULT_TABLE table=default.Table_a7e1ecd9b8b44627a65e244546a87c58 [id=56ead7419d654e5dad1631c264ec87a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.292639 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e35e0981e649f684caa96499040c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.301122 4023 catalog_manager.cc:5671] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a7356e99e744525861e12c108cfa3fe (127.3.190.1). New cstate: current_term: 1 leader_uuid: "5a7356e99e744525861e12c108cfa3fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:48.306167 4514 tablet_service.cc:1511] Processing CreateTablet for tablet db0efaecfef44457a5128e878796c061 (DEFAULT_TABLE table=default.Table_b4bb392178ef4bdbbcc1375b8497375e [id=d4bfa57fdfc64108ba1e909d0230fb51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.307212 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0efaecfef44457a5128e878796c061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.311674 4026 master_service.cc:954] ReplaceTablet: received request to replace tablet 76073435ab7949a0bb0606e6efaa48da from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:48.311796 4198 tablet_service.cc:1511] Processing CreateTablet for tablet db0efaecfef44457a5128e878796c061 (DEFAULT_TABLE table=default.Table_b4bb392178ef4bdbbcc1375b8497375e [id=d4bfa57fdfc64108ba1e909d0230fb51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.313678 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 (DEFAULT_TABLE table=default.Table_cb48b45e8d654862861b868ea8fd0368 [id=49febbf4c6a74d18bc4e1d495d1f8890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.313884 4352 tablet_service.cc:1511] Processing CreateTablet for tablet 70d410f3b3b04e649b79dda04c0891ee (DEFAULT_TABLE table=default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.314729 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395f97fdf3834cb9aaabd261e1bf76c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.315284 4358 tablet_service.cc:1511] Processing CreateTablet for tablet db0efaecfef44457a5128e878796c061 (DEFAULT_TABLE table=default.Table_b4bb392178ef4bdbbcc1375b8497375e [id=d4bfa57fdfc64108ba1e909d0230fb51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.315478 4197 tablet_service.cc:1511] Processing CreateTablet for tablet 70d410f3b3b04e649b79dda04c0891ee (DEFAULT_TABLE table=default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.316232 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0efaecfef44457a5128e878796c061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.316838 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70d410f3b3b04e649b79dda04c0891ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.260618 4818 tablet_bootstrap.cc:654] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.327503 4028 master_service.cc:954] ReplaceTablet: received request to replace tablet 2af571a9ee484ae5944d787def240c6e from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:48.310384 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 51e35e0981e649f684caa96499040c15 (DEFAULT_TABLE table=default.Table_a7e1ecd9b8b44627a65e244546a87c58 [id=56ead7419d654e5dad1631c264ec87a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.329695 4737 ts_tablet_manager.cc:1929] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:48.330144 4737 log.cc:1199] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/d8e845c461084e33802aafa9c4d91e3e
I20260430 19:58:48.330948 4737 ts_tablet_manager.cc:1950] T d8e845c461084e33802aafa9c4d91e3e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:48.331050 4198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0efaecfef44457a5128e878796c061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.332834 4223 raft_consensus.cc:3060] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:48.334440 4197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70d410f3b3b04e649b79dda04c0891ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.342924 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e35e0981e649f684caa96499040c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.362847 4831 raft_consensus.cc:493] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:48.363320 4831 raft_consensus.cc:515] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.365100 4831 leader_election.cc:290] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:48.375635 4776 tablet_bootstrap.cc:654] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.376539 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_16cd323d5c83450f8e84d0eb40d80d10" } modify_external_catalogs: true
I20260430 19:58:48.377024 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_16cd323d5c83450f8e84d0eb40d80d10" } modify_external_catalogs: true
I20260430 19:58:48.391422 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:48.391561 4744 consensus_queue.cc:1048] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:58:48.392992 4161 leader_election.cc:343] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:48.406767 4020 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad2ce1d43f214c8d9c1a1001baf53440 consensus state has no leader
I20260430 19:58:48.407194 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_9312a99c3d0040e3a6c62e6400b06041"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:48.409011 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9312a99c3d0040e3a6c62e6400b06041 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.427651 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:48.430279 4317 consensus_peers.cc:597] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e64ef047d8c46e9ac8387feb0dbe9ba. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:48.433355 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 70d410f3b3b04e649b79dda04c0891ee (DEFAULT_TABLE table=default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.433707 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 (DEFAULT_TABLE table=default.Table_cb48b45e8d654862861b868ea8fd0368 [id=49febbf4c6a74d18bc4e1d495d1f8890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.434610 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet d8e845c461084e33802aafa9c4d91e3e (table default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]) successfully deleted
I20260430 19:58:48.269742 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 56) took 1406 ms. Trace:
I20260430 19:58:48.435848 4654 rpcz_store.cc:276] 0430 19:58:46.863491 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.863608 (+ 117us) service_pool.cc:225] Handling call
0430 19:58:47.892614 (+1029006us) ts_tablet_manager.cc:1151] Deleting tablet d8e845c461084e33802aafa9c4d91e3e
0430 19:58:47.892652 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:47.961233 (+ 68581us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.266727 (+305494us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.269682 (+ 2955us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1028356}
I20260430 19:58:48.437409 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet d8e845c461084e33802aafa9c4d91e3e (table default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]) successfully deleted
W20260430 19:58:48.438419 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet e4e87fda401047cfa7070ae95b1701a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4e87fda401047cfa7070ae95b1701a9 already in progress: creating tablet
I20260430 19:58:48.440448 4729 tablet_bootstrap.cc:492] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:48.440938 4729 ts_tablet_manager.cc:1403] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.225s user 0.010s sys 0.008s
W20260430 19:58:48.443377 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 13e8206ac4484abfa769276c83fd3333 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13e8206ac4484abfa769276c83fd3333 already in progress: creating tablet
I20260430 19:58:48.443266 4729 raft_consensus.cc:359] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.443835 4729 raft_consensus.cc:385] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.444077 4729 raft_consensus.cc:740] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.444628 4729 consensus_queue.cc:260] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.445111 4020 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:48.445747 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70d410f3b3b04e649b79dda04c0891ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.449791 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395f97fdf3834cb9aaabd261e1bf76c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.455487 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 57) took 1580 ms. Trace:
I20260430 19:58:48.455832 4654 rpcz_store.cc:276] 0430 19:58:46.875189 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.875330 (+ 141us) service_pool.cc:225] Handling call
0430 19:58:48.455137 (+1579807us) ts_tablet_manager.cc:1151] Deleting tablet a7b625afadd744dc86d7bfe1da77876d
0430 19:58:48.455183 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.455426 (+ 243us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1579274}
W20260430 19:58:48.456947 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet a7b625afadd744dc86d7bfe1da77876d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7b625afadd744dc86d7bfe1da77876d already in progress: creating tablet
I20260430 19:58:48.457767 4654 tablet_replica.cc:333] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:48.458407 4654 raft_consensus.cc:2243] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:48.458892 4654 raft_consensus.cc:2272] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.459009 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 395f97fdf3834cb9aaabd261e1bf76c5
I20260430 19:58:48.463754 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:48.468499 4360 tablet_service.cc:1558] Processing DeleteTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.472645 4818 tablet_bootstrap.cc:492] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:48.473268 4818 ts_tablet_manager.cc:1403] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.509s user 0.002s sys 0.010s
I20260430 19:58:48.476388 4818 raft_consensus.cc:359] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.478281 4729 ts_tablet_manager.cc:1434] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.037s user 0.004s sys 0.000s
I20260430 19:58:48.479112 4729 tablet_bootstrap.cc:492] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
W20260430 19:58:48.481778 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 395f97fdf3834cb9aaabd261e1bf76c5 because the tablet was not found. No further retry: Not found: Tablet not found: 395f97fdf3834cb9aaabd261e1bf76c5
W20260430 19:58:48.486397 4014 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet c78d50690da7428fadb848dadefec47c consensus state has no leader
I20260430 19:58:48.485287 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 3423379d1e3d4020a4241bf8917f1050 (DEFAULT_TABLE table=default.Table_d13179c843c54969b98111d67f829f5f [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.487272 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3423379d1e3d4020a4241bf8917f1050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.490828 4503 tablet_service.cc:1511] Processing CreateTablet for tablet 081a060c44924bcda09ffc8fe88012b3 (DEFAULT_TABLE table=default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.491926 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 081a060c44924bcda09ffc8fe88012b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.486232 4513 tablet_service.cc:1511] Processing CreateTablet for tablet b6acec54bc7b4692894333e0150a1aff (DEFAULT_TABLE table=default.Table_16cd323d5c83450f8e84d0eb40d80d10 [id=9d892cfdee0b4827bed9dee0ae6761cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.495735 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6acec54bc7b4692894333e0150a1aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.500391 4729 tablet_bootstrap.cc:654] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.491798 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 081a060c44924bcda09ffc8fe88012b3 (DEFAULT_TABLE table=default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:48.503680 4158 leader_election.cc:343] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:48.508879 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 081a060c44924bcda09ffc8fe88012b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.509671 4158 leader_election.cc:304] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:48.510339 4831 raft_consensus.cc:2749] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:48.493314 4361 tablet_service.cc:1511] Processing CreateTablet for tablet b6acec54bc7b4692894333e0150a1aff (DEFAULT_TABLE table=default.Table_16cd323d5c83450f8e84d0eb40d80d10 [id=9d892cfdee0b4827bed9dee0ae6761cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.513026 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6acec54bc7b4692894333e0150a1aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.513269 4196 tablet_service.cc:1511] Processing CreateTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 (DEFAULT_TABLE table=default.Table_cb48b45e8d654862861b868ea8fd0368 [id=49febbf4c6a74d18bc4e1d495d1f8890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.514303 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395f97fdf3834cb9aaabd261e1bf76c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.514848 4191 tablet_service.cc:1558] Processing DeleteTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:48.515902 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 3423379d1e3d4020a4241bf8917f1050 (DEFAULT_TABLE table=default.Table_d13179c843c54969b98111d67f829f5f [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.516840 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3423379d1e3d4020a4241bf8917f1050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.519353 4193 tablet_service.cc:1511] Processing CreateTablet for tablet b6acec54bc7b4692894333e0150a1aff (DEFAULT_TABLE table=default.Table_16cd323d5c83450f8e84d0eb40d80d10 [id=9d892cfdee0b4827bed9dee0ae6761cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.520315 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6acec54bc7b4692894333e0150a1aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.523917 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 081a060c44924bcda09ffc8fe88012b3 (DEFAULT_TABLE table=default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.524756 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 081a060c44924bcda09ffc8fe88012b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.494411 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 3423379d1e3d4020a4241bf8917f1050 (DEFAULT_TABLE table=default.Table_d13179c843c54969b98111d67f829f5f [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:48.527344 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3423379d1e3d4020a4241bf8917f1050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:48.528164 4776 tablet_bootstrap.cc:492] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:48.528923 4776 ts_tablet_manager.cc:1403] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.256s user 0.011s sys 0.008s
I20260430 19:58:48.531078 4776 raft_consensus.cc:359] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.532847 4818 raft_consensus.cc:385] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.533201 4818 raft_consensus.cc:740] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.533896 4818 consensus_queue.cc:260] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.536319 4818 ts_tablet_manager.cc:1434] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.063s user 0.005s sys 0.000s
I20260430 19:58:48.507910 4654 ts_tablet_manager.cc:1916] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.559415 4818 tablet_bootstrap.cc:492] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
W20260430 19:58:48.564349 4029 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d4e34f3b1bf84f72855cf87683c5e318 consensus state has no leader
I20260430 19:58:48.574919 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 76073435ab7949a0bb0606e6efaa48da
I20260430 19:58:48.576262 4026 catalog_manager.cc:6741] ReplaceTablet: tablet 76073435ab7949a0bb0606e6efaa48da deleted and replaced by tablet e12d0f56f8db41aa9a1ba3f23e6428b5
I20260430 19:58:48.541255 4742 ts_tablet_manager.cc:1929] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:48.574827 4831 raft_consensus.cc:493] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:48.577471 4776 raft_consensus.cc:385] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.577737 4776 raft_consensus.cc:740] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.577658 4831 raft_consensus.cc:515] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.578549 4776 consensus_queue.cc:260] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.579533 4831 leader_election.cc:290] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:48.580943 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36490392a404b6bba29bd6803274b36" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:48.582439 4161 leader_election.cc:343] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:48.583650 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 76073435ab7949a0bb0606e6efaa48da with delete_type TABLET_DATA_DELETED (replaced by tablet e12d0f56f8db41aa9a1ba3f23e6428b5) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.583864 4742 log.cc:1199] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/d8e845c461084e33802aafa9c4d91e3e
I20260430 19:58:48.585824 4742 ts_tablet_manager.cc:1950] T d8e845c461084e33802aafa9c4d91e3e P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:48.588438 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 76073435ab7949a0bb0606e6efaa48da with delete_type TABLET_DATA_DELETED (replaced by tablet e12d0f56f8db41aa9a1ba3f23e6428b5) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:48.589190 4737 tablet_replica.cc:333] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:48.589826 4737 raft_consensus.cc:2243] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:48.590306 4737 raft_consensus.cc:2272] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.591167 4818 tablet_bootstrap.cc:654] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.592140 4737 ts_tablet_manager.cc:1916] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.580814 4776 ts_tablet_manager.cc:1434] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.052s user 0.004s sys 0.000s
I20260430 19:58:48.595013 4776 tablet_bootstrap.cc:492] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:48.601222 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 53) took 1740 ms. Trace:
I20260430 19:58:48.601802 4742 rpcz_store.cc:276] 0430 19:58:46.860803 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.861292 (+ 489us) service_pool.cc:225] Handling call
0430 19:58:48.278468 (+1417176us) ts_tablet_manager.cc:1151] Deleting tablet d8e845c461084e33802aafa9c4d91e3e
0430 19:58:48.278509 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.417811 (+139302us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.541129 (+123318us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.601161 (+ 60032us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14663,"tablet-delete.queue_time_us":1408961}
I20260430 19:58:48.603102 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 54) took 1735 ms. Trace:
I20260430 19:58:48.603442 4742 rpcz_store.cc:276] 0430 19:58:46.867503 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:46.868490 (+ 987us) service_pool.cc:225] Handling call
0430 19:58:48.602821 (+1734331us) ts_tablet_manager.cc:1151] Deleting tablet a7b625afadd744dc86d7bfe1da77876d
0430 19:58:48.602855 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.603055 (+ 200us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1733079}
I20260430 19:58:48.604377 4742 tablet_replica.cc:333] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:48.605026 4742 raft_consensus.cc:2243] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:48.606040 4742 raft_consensus.cc:2272] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.606987 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet d8e845c461084e33802aafa9c4d91e3e (table default.Table_0f760a2ab2ce43659a9fcbc0ea85758e [id=e5805fa02a04405fb9587fc43e667afa]) successfully deleted
W20260430 19:58:48.607590 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet a7b625afadd744dc86d7bfe1da77876d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7b625afadd744dc86d7bfe1da77876d already in progress: creating tablet
I20260430 19:58:48.609299 4742 ts_tablet_manager.cc:1916] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.609727 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 76073435ab7949a0bb0606e6efaa48da with delete_type TABLET_DATA_DELETED (replaced by tablet e12d0f56f8db41aa9a1ba3f23e6428b5) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:48.628607 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_2a59def4cb444084bf2b70663d1430b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:48.629690 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36490392a404b6bba29bd6803274b36" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:48.630280 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a59def4cb444084bf2b70663d1430b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.630272 4532 raft_consensus.cc:2468] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:48.623059 4654 ts_tablet_manager.cc:1929] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:48.632743 4654 log.cc:1199] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/2af571a9ee484ae5944d787def240c6e
I20260430 19:58:48.632473 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_acaae4cbbb6443049cb4c80749caf848"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:48.635200 4654 ts_tablet_manager.cc:1950] T 2af571a9ee484ae5944d787def240c6e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
W20260430 19:58:48.635926 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acaae4cbbb6443049cb4c80749caf848 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.637426 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 67) took 1174 ms. Trace:
I20260430 19:58:48.648898 4654 rpcz_store.cc:276] 0430 19:58:47.463020 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.463154 (+ 134us) service_pool.cc:225] Handling call
0430 19:58:48.457625 (+994471us) ts_tablet_manager.cc:1151] Deleting tablet 2af571a9ee484ae5944d787def240c6e
0430 19:58:48.457662 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.568529 (+110867us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.622934 (+ 54405us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.637369 (+ 14435us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":993864}
I20260430 19:58:48.650039 4158 leader_election.cc:304] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:48.650379 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 2af571a9ee484ae5944d787def240c6e (table default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]) successfully deleted
I20260430 19:58:48.648563 4818 tablet_bootstrap.cc:492] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:48.651357 4818 ts_tablet_manager.cc:1403] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.092s user 0.006s sys 0.006s
I20260430 19:58:48.651373 4831 raft_consensus.cc:2804] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:48.651741 4831 raft_consensus.cc:493] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:48.652032 4831 raft_consensus.cc:3060] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:48.652068 4769 raft_consensus.cc:493] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:48.652552 4769 raft_consensus.cc:515] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.654031 4776 tablet_bootstrap.cc:654] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.654392 4769 leader_election.cc:290] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:48.655157 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 73) took 1067 ms. Trace:
I20260430 19:58:48.656208 4654 rpcz_store.cc:276] 0430 19:58:47.588007 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.588879 (+ 872us) service_pool.cc:225] Handling call
0430 19:58:48.654815 (+1065936us) ts_tablet_manager.cc:1151] Deleting tablet 6220177b3e3e4d18bd8e20e6d9d96be7
0430 19:58:48.654854 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.655094 (+ 240us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1065309}
I20260430 19:58:48.656558 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d1b860c7073446597ce696f822ce1c2" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:48.655599 4818 raft_consensus.cc:359] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.657227 4383 raft_consensus.cc:2393] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate 5a7356e99e744525861e12c108cfa3fe in this term.
I20260430 19:58:48.657284 4818 raft_consensus.cc:385] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.657603 4818 raft_consensus.cc:740] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
W20260430 19:58:48.658340 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6220177b3e3e4d18bd8e20e6d9d96be7 already in progress: creating tablet
I20260430 19:58:48.661737 4831 raft_consensus.cc:515] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.659376 4654 tablet_replica.cc:333] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:48.663489 4654 raft_consensus.cc:2243] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:48.669257 4654 raft_consensus.cc:2272] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.664345 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36490392a404b6bba29bd6803274b36" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:48.664026 4818 consensus_queue.cc:260] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.670195 4383 raft_consensus.cc:3060] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:48.677398 4383 raft_consensus.cc:2468] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:48.678267 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a989a7af579548079a4722aee571d7b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:48.675778 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d1b860c7073446597ce696f822ce1c2" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:48.679196 4161 leader_election.cc:304] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
W20260430 19:58:48.680154 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a989a7af579548079a4722aee571d7b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:48.681927 4470 leader_election.cc:304] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:48.683044 4769 raft_consensus.cc:3060] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:48.686736 4829 raft_consensus.cc:2804] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:48.692240 4829 raft_consensus.cc:697] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:48.693297 4769 raft_consensus.cc:2749] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:48.693327 4829 consensus_queue.cc:237] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.695566 4818 ts_tablet_manager.cc:1434] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.044s user 0.005s sys 0.001s
I20260430 19:58:48.706993 4654 ts_tablet_manager.cc:1916] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.713802 4818 tablet_bootstrap.cc:492] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:48.717480 4021 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_a2efa2a9f9bd42738350b731f2debc69" } new_table_name: "default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f" modify_external_catalogs: true
I20260430 19:58:48.719033 4027 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_276bd93958054b6b9ad30f314bbb487c" } modify_external_catalogs: true
I20260430 19:58:48.719518 4027 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_276bd93958054b6b9ad30f314bbb487c" } modify_external_catalogs: true
I20260430 19:58:48.722249 4831 leader_election.cc:290] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:48.722939 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36490392a404b6bba29bd6803274b36" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:48.723469 4532 raft_consensus.cc:3060] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:48.727309 4024 master_service.cc:954] ReplaceTablet: received request to replace tablet 0e62335e43c1410b8b3acdd9e2f47981 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:48.732918 4360 tablet_service.cc:1558] Processing DeleteTablet for tablet a7b625afadd744dc86d7bfe1da77876d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.733803 4737 ts_tablet_manager.cc:1929] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:48.756487 4776 tablet_bootstrap.cc:492] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:48.756942 4818 tablet_bootstrap.cc:654] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.756994 4776 ts_tablet_manager.cc:1403] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.162s user 0.013s sys 0.016s
I20260430 19:58:48.759456 4776 raft_consensus.cc:359] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:48.760118 4776 raft_consensus.cc:385] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.760409 4776 raft_consensus.cc:740] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.761090 4776 consensus_queue.cc:260] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:48.762951 4776 ts_tablet_manager.cc:1434] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260430 19:58:48.769400 4776 tablet_bootstrap.cc:492] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:48.779886 4776 tablet_bootstrap.cc:654] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.784361 4654 ts_tablet_manager.cc:1929] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:48.784790 4654 log.cc:1199] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/cfb8e475e89f434f98482fee1b45dbca
I20260430 19:58:48.785431 4654 ts_tablet_manager.cc:1950] T cfb8e475e89f434f98482fee1b45dbca P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:48.787576 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet b6acec54bc7b4692894333e0150a1aff
I20260430 19:58:48.801397 4196 tablet_service.cc:1558] Processing DeleteTablet for tablet b6acec54bc7b4692894333e0150a1aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:48.804455 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 74) took 1216 ms. Trace:
I20260430 19:58:48.734544 4532 raft_consensus.cc:2468] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:48.749817 4729 tablet_bootstrap.cc:492] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:48.804803 4654 rpcz_store.cc:276] 0430 19:58:47.588372 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.589660 (+ 1288us) service_pool.cc:225] Handling call
0430 19:58:48.659208 (+1069548us) ts_tablet_manager.cc:1151] Deleting tablet cfb8e475e89f434f98482fee1b45dbca
0430 19:58:48.659251 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.747469 (+ 88218us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.784235 (+ 36766us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.804389 (+ 20154us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1036497}
I20260430 19:58:48.805577 4729 ts_tablet_manager.cc:1403] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.327s user 0.033s sys 0.065s
I20260430 19:58:48.805811 4654 tablet_replica.cc:333] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:48.805877 4737 log.cc:1199] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/76073435ab7949a0bb0606e6efaa48da
I20260430 19:58:48.805583 4028 catalog_manager.cc:5671] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a7356e99e744525861e12c108cfa3fe (127.3.190.1). New cstate: current_term: 1 leader_uuid: "5a7356e99e744525861e12c108cfa3fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:48.806614 4737 ts_tablet_manager.cc:1950] T 76073435ab7949a0bb0606e6efaa48da P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:48.807035 4028 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:48.808106 4729 raft_consensus.cc:359] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.808697 4729 raft_consensus.cc:385] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.808935 4729 raft_consensus.cc:740] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.809764 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet cfb8e475e89f434f98482fee1b45dbca (table default.Table_d13179c843c54969b98111d67f829f5f [id=37e26a1d9e4f40cc8fb14b9934ba3780]) successfully deleted
I20260430 19:58:48.809505 4729 consensus_queue.cc:260] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.813652 4729 ts_tablet_manager.cc:1434] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260430 19:58:48.814513 4729 tablet_bootstrap.cc:492] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:48.820076 4818 tablet_bootstrap.cc:492] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:48.820276 4729 tablet_bootstrap.cc:654] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.820618 4818 ts_tablet_manager.cc:1403] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.113s user 0.007s sys 0.010s
I20260430 19:58:48.823197 4818 raft_consensus.cc:359] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:48.823879 4818 raft_consensus.cc:385] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.824173 4818 raft_consensus.cc:740] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.824843 4818 consensus_queue.cc:260] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:48.831900 4818 ts_tablet_manager.cc:1434] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260430 19:58:48.832805 4818 tablet_bootstrap.cc:492] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:48.833446 4729 tablet_bootstrap.cc:492] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:48.834865 4729 ts_tablet_manager.cc:1403] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.000s
I20260430 19:58:48.837728 4818 tablet_bootstrap.cc:654] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.841883 4027 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 09f272340cd942e9b2af9e38a2bdec82
W20260430 19:58:48.842074 4021 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4764dc427b4a48309de914c4f5e655fe consensus state has no leader
I20260430 19:58:48.842984 4729 raft_consensus.cc:359] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.844367 4196 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:48.844590 4729 raft_consensus.cc:385] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.845197 4729 raft_consensus.cc:740] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.846093 4729 consensus_queue.cc:260] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.849184 4729 ts_tablet_manager.cc:1434] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260430 19:58:48.851459 4729 tablet_bootstrap.cc:492] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:48.857543 4742 ts_tablet_manager.cc:1929] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:48.863844 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.870254 4360 tablet_service.cc:1558] Processing DeleteTablet for tablet b6acec54bc7b4692894333e0150a1aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.870483 4729 tablet_bootstrap.cc:654] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.874375 4742 log.cc:1199] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/2af571a9ee484ae5944d787def240c6e
I20260430 19:58:48.875612 4742 ts_tablet_manager.cc:1950] T 2af571a9ee484ae5944d787def240c6e P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:48.879940 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 61) took 1401 ms. Trace:
I20260430 19:58:48.881255 4742 rpcz_store.cc:276] 0430 19:58:47.478997 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.511295 (+ 32298us) service_pool.cc:225] Handling call
0430 19:58:48.604254 (+1092959us) ts_tablet_manager.cc:1151] Deleting tablet 2af571a9ee484ae5944d787def240c6e
0430 19:58:48.604286 (+ 32us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.746159 (+141873us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.857401 (+111242us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.879880 (+ 22479us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091849}
I20260430 19:58:48.886121 4818 tablet_bootstrap.cc:492] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:48.886972 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 2af571a9ee484ae5944d787def240c6e (table default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]) successfully deleted
I20260430 19:58:48.888393 4742 tablet_replica.cc:333] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:48.890086 4729 tablet_bootstrap.cc:492] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:48.890471 4729 ts_tablet_manager.cc:1403] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.039s user 0.011s sys 0.000s
I20260430 19:58:48.890970 4818 ts_tablet_manager.cc:1403] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.002s
I20260430 19:58:48.891695 4742 raft_consensus.cc:2243] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:48.893137 4729 raft_consensus.cc:359] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.893729 4729 raft_consensus.cc:385] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.893958 4729 raft_consensus.cc:740] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.896363 4818 raft_consensus.cc:359] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.897357 4742 raft_consensus.cc:2272] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.894471 4729 consensus_queue.cc:260] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.900677 4742 ts_tablet_manager.cc:1916] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.906276 4818 raft_consensus.cc:385] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.907063 4818 raft_consensus.cc:740] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.907714 4729 ts_tablet_manager.cc:1434] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260430 19:58:48.909582 4729 tablet_bootstrap.cc:492] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:48.909130 4818 consensus_queue.cc:260] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:48.927049 4729 tablet_bootstrap.cc:654] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.937026 4654 raft_consensus.cc:2243] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:48.938712 4654 raft_consensus.cc:2272] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:48.941242 4729 tablet_bootstrap.cc:492] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:48.943190 4654 ts_tablet_manager.cc:1916] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:48.946959 4729 ts_tablet_manager.cc:1403] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.000s
I20260430 19:58:48.949363 4729 raft_consensus.cc:359] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:48.950016 4729 raft_consensus.cc:385] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:48.950258 4729 raft_consensus.cc:740] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:48.954264 4818 ts_tablet_manager.cc:1434] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.060s user 0.006s sys 0.005s
I20260430 19:58:48.955480 4729 consensus_queue.cc:260] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:48.959252 4818 tablet_bootstrap.cc:492] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:48.965442 4818 tablet_bootstrap.cc:654] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.968305 4023 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_d904dde10c354ae5ab6b325322b1c0ae" } new_table_name: "default.Table_7915db2fd4cb498483b1c905765b8ef8" modify_external_catalogs: true
I20260430 19:58:48.960960 4729 ts_tablet_manager.cc:1434] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260430 19:58:48.975384 4729 tablet_bootstrap.cc:492] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:48.980687 4729 tablet_bootstrap.cc:654] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:48.970106 4024 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 0e62335e43c1410b8b3acdd9e2f47981
I20260430 19:58:48.982781 4024 catalog_manager.cc:6741] ReplaceTablet: tablet 0e62335e43c1410b8b3acdd9e2f47981 deleted and replaced by tablet cfd78a1cd7a94e27bb376d1b7c151073
I20260430 19:58:48.970571 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:48.989408 4360 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 with delete_type TABLET_DATA_DELETED (replaced by tablet cfd78a1cd7a94e27bb376d1b7c151073) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:48.994055 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:48.994786 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:48.995857 4818 tablet_bootstrap.cc:492] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:48.996271 4818 ts_tablet_manager.cc:1403] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.000s
I20260430 19:58:49.022997 4196 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 with delete_type TABLET_DATA_DELETED (replaced by tablet cfd78a1cd7a94e27bb376d1b7c151073) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:49.049862 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]) successfully deleted
I20260430 19:58:49.059700 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:49.059698 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 with delete_type TABLET_DATA_DELETED (replaced by tablet cfd78a1cd7a94e27bb376d1b7c151073) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:49.059698 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet b6acec54bc7b4692894333e0150a1aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:49.071029 4317 consensus_peers.cc:597] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76073435ab7949a0bb0606e6efaa48da. This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:49.074362 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 09f272340cd942e9b2af9e38a2bdec82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09f272340cd942e9b2af9e38a2bdec82 already in progress: creating tablet
W20260430 19:58:49.075083 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 0e62335e43c1410b8b3acdd9e2f47981 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e62335e43c1410b8b3acdd9e2f47981 already in progress: creating tablet
I20260430 19:58:49.075121 4769 raft_consensus.cc:493] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260430 19:58:49.075639 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet b6acec54bc7b4692894333e0150a1aff because the tablet was not found. No further retry: Not found: Tablet not found: b6acec54bc7b4692894333e0150a1aff
I20260430 19:58:49.075763 4769 raft_consensus.cc:515] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:49.078127 4769 leader_election.cc:290] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:49.079152 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78d50690da7428fadb848dadefec47c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:49.079779 4383 raft_consensus.cc:2468] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:49.081187 4470 leader_election.cc:304] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:49.082119 4769 raft_consensus.cc:2804] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:49.082533 4769 raft_consensus.cc:493] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:49.082942 4769 raft_consensus.cc:3060] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.087808 4742 ts_tablet_manager.cc:1929] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:49.088179 4742 log.cc:1199] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/cfb8e475e89f434f98482fee1b45dbca
I20260430 19:58:49.088811 4742 ts_tablet_manager.cc:1950] T cfb8e475e89f434f98482fee1b45dbca P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:49.089548 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 69) took 1371 ms. Trace:
I20260430 19:58:49.089847 4742 rpcz_store.cc:276] 0430 19:58:47.717664 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.717827 (+ 163us) service_pool.cc:225] Handling call
0430 19:58:48.888225 (+1170398us) ts_tablet_manager.cc:1151] Deleting tablet cfb8e475e89f434f98482fee1b45dbca
0430 19:58:48.888267 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.020449 (+132182us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.087675 (+ 67226us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.089494 (+ 1819us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1123210}
I20260430 19:58:49.093748 4769 raft_consensus.cc:515] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:49.095224 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78d50690da7428fadb848dadefec47c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:49.095800 4225 raft_consensus.cc:2468] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:49.096258 4023 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_7e8d384dfa62442fbf585d952b02fe29" } modify_external_catalogs: true
I20260430 19:58:49.096954 4023 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_7e8d384dfa62442fbf585d952b02fe29" } modify_external_catalogs: true
I20260430 19:58:49.097641 4383 raft_consensus.cc:1275] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Refusing update from remote peer 5a7356e99e744525861e12c108cfa3fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:49.099074 4870 consensus_queue.cc:1048] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:58:49.105902 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet cfb8e475e89f434f98482fee1b45dbca (table default.Table_d13179c843c54969b98111d67f829f5f [id=37e26a1d9e4f40cc8fb14b9934ba3780]) successfully deleted
I20260430 19:58:49.120635 4776 tablet_bootstrap.cc:492] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:49.121052 4776 ts_tablet_manager.cc:1403] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.352s user 0.058s sys 0.076s
I20260430 19:58:49.123347 4776 raft_consensus.cc:359] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.123962 4776 raft_consensus.cc:385] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.124233 4776 raft_consensus.cc:740] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.124784 4776 consensus_queue.cc:260] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.128681 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_81ac9df8ed4947d8bae7f45d6dfef943"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:49.131994 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_7915db2fd4cb498483b1c905765b8ef8" } new_table_name: "default.Table_028cf8f724c64a51adefa0cc9c15a50e" modify_external_catalogs: true
I20260430 19:58:49.136148 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_308a505e52094e64933fe14eee977c7b" } modify_external_catalogs: true
I20260430 19:58:49.136621 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_308a505e52094e64933fe14eee977c7b" } modify_external_catalogs: true
I20260430 19:58:49.137008 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_830b99b7bf1e49e8ad90881ae777dca7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:49.126482 4776 ts_tablet_manager.cc:1434] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260430 19:58:49.137850 4027 master_service.cc:954] ReplaceTablet: received request to replace tablet e36490392a404b6bba29bd6803274b36 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:49.138324 4776 tablet_bootstrap.cc:492] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
W20260430 19:58:49.138564 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_830b99b7bf1e49e8ad90881ae777dca7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.138638 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_8ebf726da3884df1b2b6805cb6b147a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:49.139959 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ebf726da3884df1b2b6805cb6b147a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.145094 4885 raft_consensus.cc:493] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260430 19:58:49.130360 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81ac9df8ed4947d8bae7f45d6dfef943 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.145529 4885 raft_consensus.cc:515] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.147384 4885 leader_election.cc:290] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:49.151024 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b625afadd744dc86d7bfe1da77876d" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:49.155252 4776 tablet_bootstrap.cc:654] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:49.155483 4320 leader_election.cc:343] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.022380 4654 ts_tablet_manager.cc:1929] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:49.158918 4654 log.cc:1199] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/0e64ef047d8c46e9ac8387feb0dbe9ba
I20260430 19:58:49.159623 4654 ts_tablet_manager.cc:1950] T 0e64ef047d8c46e9ac8387feb0dbe9ba P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:49.161434 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 76) took 1519 ms. Trace:
I20260430 19:58:49.170095 4776 tablet_bootstrap.cc:492] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:49.170499 4776 ts_tablet_manager.cc:1403] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.000s
I20260430 19:58:49.172792 4776 raft_consensus.cc:359] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.173394 4776 raft_consensus.cc:385] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.173646 4776 raft_consensus.cc:740] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.174724 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b625afadd744dc86d7bfe1da77876d" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:49.175523 4532 raft_consensus.cc:2468] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:49.179293 4317 leader_election.cc:304] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:49.180161 4885 raft_consensus.cc:2804] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:49.174468 4769 leader_election.cc:290] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:49.180509 4885 raft_consensus.cc:493] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:49.180794 4885 raft_consensus.cc:3060] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.180830 4769 raft_consensus.cc:493] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.181242 4769 raft_consensus.cc:515] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.174168 4776 consensus_queue.cc:260] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.182940 4769 leader_election.cc:290] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:49.187145 4776 ts_tablet_manager.cc:1434] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260430 19:58:49.187304 4885 raft_consensus.cc:515] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.188010 4776 tablet_bootstrap.cc:492] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:49.189743 4885 leader_election.cc:290] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:49.191669 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b625afadd744dc86d7bfe1da77876d" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:49.192229 4225 raft_consensus.cc:3060] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.194851 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7b625afadd744dc86d7bfe1da77876d" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:49.195391 4532 raft_consensus.cc:3060] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.203167 4776 tablet_bootstrap.cc:654] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.207671 4871 consensus_queue.cc:1048] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:58:49.210650 4225 raft_consensus.cc:2468] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:49.206276 4532 raft_consensus.cc:2468] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:49.215735 4729 tablet_bootstrap.cc:492] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:49.216257 4729 ts_tablet_manager.cc:1403] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.241s user 0.031s sys 0.026s
I20260430 19:58:49.219506 4729 raft_consensus.cc:359] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.220171 4729 raft_consensus.cc:385] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.220427 4729 raft_consensus.cc:740] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.220983 4729 consensus_queue.cc:260] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.222973 4729 ts_tablet_manager.cc:1434] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260430 19:58:49.223904 4729 tablet_bootstrap.cc:492] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:49.225292 4742 tablet_replica.cc:333] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:49.225953 4742 raft_consensus.cc:2243] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:49.226402 4742 raft_consensus.cc:2272] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.229882 4729 tablet_bootstrap.cc:654] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.233186 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_f0911e2389af4d0c90e177647db6ee84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:49.234825 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0911e2389af4d0c90e177647db6ee84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.155699 4818 raft_consensus.cc:359] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.242238 4818 raft_consensus.cc:385] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.242527 4818 raft_consensus.cc:740] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.211637 4320 leader_election.cc:304] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:49.252638 4885 raft_consensus.cc:2804] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:49.253057 4885 raft_consensus.cc:697] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:49.253800 4885 consensus_queue.cc:237] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.248090 4818 consensus_queue.cc:260] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.258333 4029 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_12fc75157d86414bac1739e26d54d739" } new_table_name: "default.Table_d87d24b7c4654f5b98b432530730c721" modify_external_catalogs: true
I20260430 19:58:49.280510 4818 ts_tablet_manager.cc:1434] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.284s user 0.007s sys 0.000s
I20260430 19:58:49.281423 4885 raft_consensus.cc:493] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.282104 4885 raft_consensus.cc:515] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.161732 4654 rpcz_store.cc:276] 0430 19:58:47.642313 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.651203 (+ 8890us) service_pool.cc:225] Handling call
0430 19:58:48.805656 (+1154453us) ts_tablet_manager.cc:1151] Deleting tablet 0e64ef047d8c46e9ac8387feb0dbe9ba
0430 19:58:48.805702 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:48.937791 (+132089us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54001301d4. stack: 00007ff08b6a8c83 00007ff08b6a8b80 00007ff08b219574 000000000050e094 000000000050e029 00007ff0926f7951 00007ff092796742 00007ff092bb97bc 00007ff093811cca 00007ff093842990 00007ff093828efd 00007ff093828ea9 00007ff093828e39 00007ff093828e01 00007ff09382802d 00000000005b3bc4
0430 19:58:48.955962 (+ 18171us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.022265 (+ 66303us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.161369 (+139104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300142976,"tablet-delete.queue_time_us":1152103}
I20260430 19:58:49.283919 4885 leader_election.cc:290] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
W20260430 19:58:49.286662 4029 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c81b11280f6d49e681537c88fb1cbf68 consensus state has no leader
I20260430 19:58:49.288319 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:49.288965 4531 raft_consensus.cc:1275] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Refusing update from remote peer 5a7356e99e744525861e12c108cfa3fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:49.292598 4532 raft_consensus.cc:2468] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:49.294270 4317 leader_election.cc:304] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:49.295058 4885 raft_consensus.cc:2804] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:49.295459 4885 raft_consensus.cc:493] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:49.295812 4885 raft_consensus.cc:3060] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.283996 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 0e64ef047d8c46e9ac8387feb0dbe9ba (table default.Table_ccf1123d6d5d4e489838f28c6125aa15 [id=007139834a09457dbfcb9a426839d894]) successfully deleted
I20260430 19:58:49.302495 4885 raft_consensus.cc:515] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.304286 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 83) took 1269 ms. Trace:
I20260430 19:58:49.304607 4654 rpcz_store.cc:276] 0430 19:58:48.034765 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.034876 (+ 111us) service_pool.cc:225] Handling call
0430 19:58:49.303924 (+1269048us) ts_tablet_manager.cc:1151] Deleting tablet e4e87fda401047cfa7070ae95b1701a9
0430 19:58:49.303970 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.304214 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1268506}
I20260430 19:58:49.309477 4742 ts_tablet_manager.cc:1916] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:44.662230 4683 debug-util.cc:398] Leaking SignalData structure 0x7b0800034ca0 after lost signal to thread 4635
W20260430 19:58:49.352927 4683 kernel_stack_watchdog.cc:198] Thread 4635 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 349ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260430 19:58:43.312832 4689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:49.310290 4871 consensus_queue.cc:1048] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260430 19:58:49.318862 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:49.359870 4222 raft_consensus.cc:3060] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.370301 4222 raft_consensus.cc:2468] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:49.329938 4891 tablet.cc:1724] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:49.331705 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:49.342002 4776 tablet_bootstrap.cc:492] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:49.372782 4225 raft_consensus.cc:2376] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1. Re-sending same reply.
I20260430 19:58:49.372880 4776 ts_tablet_manager.cc:1403] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.185s user 0.029s sys 0.023s
I20260430 19:58:49.375597 4776 raft_consensus.cc:359] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.376286 4776 raft_consensus.cc:385] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.376545 4776 raft_consensus.cc:740] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.377171 4776 consensus_queue.cc:260] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.379542 4776 ts_tablet_manager.cc:1434] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260430 19:58:49.358544 4635 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 6.048s user 0.443s sys 0.641s
W20260430 19:58:49.380189 4635 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 6.070s user 0.443s sys 0.642s
I20260430 19:58:49.380357 4776 tablet_bootstrap.cc:492] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:49.305476 4885 leader_election.cc:290] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:49.387768 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 84) took 1251 ms. Trace:
I20260430 19:58:49.308380 4890 tablet.cc:1724] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:49.388959 4635 server_base.cc:1061] running on GCE node
I20260430 19:58:49.371403 4320 leader_election.cc:304] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:49.281412 4818 tablet_bootstrap.cc:492] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
W20260430 19:58:49.398169 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet e4e87fda401047cfa7070ae95b1701a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4e87fda401047cfa7070ae95b1701a9 already in progress: creating tablet
I20260430 19:58:49.398751 4885 raft_consensus.cc:2804] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:49.399209 4885 raft_consensus.cc:697] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:49.399976 4885 consensus_queue.cc:237] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.401293 4776 tablet_bootstrap.cc:654] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.388168 4654 rpcz_store.cc:276] 0430 19:58:48.136487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.136637 (+ 150us) service_pool.cc:225] Handling call
0430 19:58:49.387512 (+1250875us) ts_tablet_manager.cc:1151] Deleting tablet 3946fffc0d2f47f789c7341dc66d603e
0430 19:58:49.387557 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.387715 (+ 158us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1250298}
I20260430 19:58:49.403978 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 86) took 1229 ms. Trace:
I20260430 19:58:49.407261 4654 rpcz_store.cc:276] 0430 19:58:48.174142 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.174256 (+ 114us) service_pool.cc:225] Handling call
0430 19:58:49.402807 (+1228551us) ts_tablet_manager.cc:1151] Deleting tablet 13e8206ac4484abfa769276c83fd3333
0430 19:58:49.402848 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.403920 (+ 1072us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1227963}
W20260430 19:58:49.409168 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 3946fffc0d2f47f789c7341dc66d603e because the tablet was not found. No further retry: Not found: Tablet not found: 3946fffc0d2f47f789c7341dc66d603e
I20260430 19:58:49.421169 4776 tablet_bootstrap.cc:492] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:49.421599 4776 ts_tablet_manager.cc:1403] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.041s user 0.009s sys 0.003s
W20260430 19:58:49.420290 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 13e8206ac4484abfa769276c83fd3333 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13e8206ac4484abfa769276c83fd3333 already in progress: creating tablet
W20260430 19:58:49.423250 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 395f97fdf3834cb9aaabd261e1bf76c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 395f97fdf3834cb9aaabd261e1bf76c5 already in progress: creating tablet
I20260430 19:58:49.423851 4776 raft_consensus.cc:359] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.424428 4776 raft_consensus.cc:385] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.424644 4776 raft_consensus.cc:740] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.425951 4776 consensus_queue.cc:260] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.426442 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:49.427443 4532 raft_consensus.cc:3060] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.427265 4901 consensus_queue.cc:1048] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260430 19:58:49.431078 4902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:58:49.428975 4776 ts_tablet_manager.cc:1434] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260430 19:58:49.434891 4776 tablet_bootstrap.cc:492] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:49.440040 4776 tablet_bootstrap.cc:654] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.446939 4635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:58:49.450565 4729 tablet_bootstrap.cc:492] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:49.451149 4729 ts_tablet_manager.cc:1403] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.227s user 0.043s sys 0.072s
I20260430 19:58:49.453613 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78d50690da7428fadb848dadefec47c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:49.453769 4729 raft_consensus.cc:359] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.454243 4225 raft_consensus.cc:3060] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.454413 4729 raft_consensus.cc:385] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.454692 4729 raft_consensus.cc:740] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.459400 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a7ec0d6efc404f0d80ccdee5fc87c213"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:49.459806 4729 consensus_queue.cc:260] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:49.461206 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7ec0d6efc404f0d80ccdee5fc87c213 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.461793 4729 ts_tablet_manager.cc:1434] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260430 19:58:49.462761 4729 tablet_bootstrap.cc:492] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:49.468062 4225 raft_consensus.cc:2468] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:49.471959 4635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:58:49.474776 4635 hybrid_clock.cc:648] HybridClock initialized: now 1777579129474723 us; error 42 us; skew 500 ppm
I20260430 19:58:49.478168 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_345c24b2a5384ad09a15536943d06a18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:49.478256 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_5b5580edb7844ccb8271ad24e96a1eb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:49.479877 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_345c24b2a5384ad09a15536943d06a18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:49.479997 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b5580edb7844ccb8271ad24e96a1eb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.475864 4635 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:58:49.500912 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c78d50690da7428fadb848dadefec47c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:49.501479 4382 raft_consensus.cc:3060] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.501538 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:49.502197 4225 raft_consensus.cc:2393] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:49.504324 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4b0d830f1b14ce382b2e0242f6565e7" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:49.509019 4532 raft_consensus.cc:2468] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
W20260430 19:58:49.513813 4014 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet c78d50690da7428fadb848dadefec47c consensus state has no leader
I20260430 19:58:49.519312 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_5bf270e5198e4ed491b5536ff5ed2629"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:49.520818 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5bf270e5198e4ed491b5536ff5ed2629 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.529917 4024 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_b4bb392178ef4bdbbcc1375b8497375e" } new_table_name: "default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c" modify_external_catalogs: true
I20260430 19:58:49.535254 4382 raft_consensus.cc:2468] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:49.539340 4654 tablet_replica.cc:333] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:49.540169 4654 raft_consensus.cc:2243] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:49.540977 4654 raft_consensus.cc:2272] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.554085 4818 tablet_bootstrap.cc:700] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent opening tablet: real 0.152s user 0.003s sys 0.000s
I20260430 19:58:49.554755 4818 tablet_bootstrap.cc:654] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:49.570127 4024 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet db0efaecfef44457a5128e878796c061 consensus state has no leader
I20260430 19:58:49.509701 4893 raft_consensus.cc:493] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.579008 4893 raft_consensus.cc:515] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.580785 4893 leader_election.cc:290] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:49.510481 4470 leader_election.cc:304] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:49.521389 4769 raft_consensus.cc:493] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.584501 4769 raft_consensus.cc:515] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.587052 4769 leader_election.cc:290] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:49.587849 4769 raft_consensus.cc:2804] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:49.588291 4769 raft_consensus.cc:697] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:49.589198 4769 consensus_queue.cc:237] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
W20260430 19:58:49.595161 4470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 84704.2us
I20260430 19:58:49.596192 4470 leader_election.cc:304] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:49.598803 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2ce1d43f214c8d9c1a1001baf53440" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:49.600548 4893 raft_consensus.cc:2749] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:49.606328 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2ce1d43f214c8d9c1a1001baf53440" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:49.608911 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa59394fc34f419ca22a695b8dff4613" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:49.609594 4225 raft_consensus.cc:2468] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:49.610915 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet e36490392a404b6bba29bd6803274b36 (table default.Table_7915db2fd4cb498483b1c905765b8ef8 [id=cd706c6a34ac49a4b82e411aca27698a]). Expected version 1 got 0
I20260430 19:58:49.611675 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa59394fc34f419ca22a695b8dff4613" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:49.612221 4382 raft_consensus.cc:2468] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
W20260430 19:58:49.619801 4470 leader_election.cc:343] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.624992 4022 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:49.627727 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 (DEFAULT_TABLE table=default.Table_74850f27abe54175909dc1d9c494804b [id=7ad67f36177f4ef18a33fa7a65cbef6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.630178 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:49.630906 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet ad2ce1d43f214c8d9c1a1001baf53440 (table default.Table_cf6f9cf5be37436f9671fa67edc046a1 [id=043f0b9bc9274b9fbdbdd17fb0d6689c]). Expected version 2 got 0
I20260430 19:58:49.642886 4654 ts_tablet_manager.cc:1916] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:49.644728 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 443308f2a26f4622aa4551ff5f73fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.630019 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 (DEFAULT_TABLE table=default.Table_74850f27abe54175909dc1d9c494804b [id=7ad67f36177f4ef18a33fa7a65cbef6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.651126 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 443308f2a26f4622aa4551ff5f73fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.635056 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet a7b625afadd744dc86d7bfe1da77876d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:49.636773 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet cfb8e475e89f434f98482fee1b45dbca with delete_type TABLET_DATA_DELETED (replaced by tablet 3423379d1e3d4020a4241bf8917f1050) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:49.656797 4915 tablet_replica.cc:333] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:49.657892 4915 raft_consensus.cc:2243] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:49.659184 4915 raft_consensus.cc:2272] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.572031 4911 raft_consensus.cc:493] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.659204 4470 leader_election.cc:304] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:49.663057 4769 raft_consensus.cc:2804] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:49.662815 4911 raft_consensus.cc:515] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.663429 4769 raft_consensus.cc:493] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:49.663779 4769 raft_consensus.cc:3060] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.664855 4911 leader_election.cc:290] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
W20260430 19:58:49.671195 4470 leader_election.cc:343] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.671659 4470 leader_election.cc:304] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
W20260430 19:58:49.673035 4470 leader_election.cc:343] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: cfb8e475e89f434f98482fee1b45dbca
I20260430 19:58:49.675549 4196 tablet_service.cc:1511] Processing CreateTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 (DEFAULT_TABLE table=default.Table_74850f27abe54175909dc1d9c494804b [id=7ad67f36177f4ef18a33fa7a65cbef6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.676610 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 443308f2a26f4622aa4551ff5f73fc39. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:49.677788 4470 leader_election.cc:343] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Not found: Tablet not found: cfb8e475e89f434f98482fee1b45dbca
I20260430 19:58:49.678150 4470 leader_election.cc:304] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:49.673579 4912 raft_consensus.cc:2749] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:49.679418 4912 raft_consensus.cc:2749] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:49.682256 4915 ts_tablet_manager.cc:1916] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:49.676189 4769 raft_consensus.cc:515] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.687561 4729 tablet_bootstrap.cc:654] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.687477 4891 tablet.cc:1724] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:49.687943 4769 leader_election.cc:290] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:49.689747 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa59394fc34f419ca22a695b8dff4613" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:49.690341 4225 raft_consensus.cc:3060] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.690907 4890 tablet.cc:1724] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:49.686208 4917 tablet.cc:1724] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:49.696990 4225 raft_consensus.cc:2468] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:49.698706 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa59394fc34f419ca22a695b8dff4613" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:49.699311 4384 raft_consensus.cc:3060] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:49.703655 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet 37d6c54ae89946158cad45e741c1c0f0 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:49.704130 4742 ts_tablet_manager.cc:1929] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:49.704541 4742 log.cc:1199] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/0e64ef047d8c46e9ac8387feb0dbe9ba
I20260430 19:58:49.705219 4742 ts_tablet_manager.cc:1950] T 0e64ef047d8c46e9ac8387feb0dbe9ba P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:49.706130 4384 raft_consensus.cc:2468] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:49.706360 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 958ba8046c294be3ac20e2d93925d17b (DEFAULT_TABLE table=default.Table_40c9009b608449f5ad959e33a2fbbc12 [id=ffb188d244694ce2aa7985340abb8813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.707347 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958ba8046c294be3ac20e2d93925d17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.708433 4729 tablet_bootstrap.cc:492] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:49.708824 4729 ts_tablet_manager.cc:1403] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.246s user 0.013s sys 0.003s
I20260430 19:58:49.709731 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 958ba8046c294be3ac20e2d93925d17b (DEFAULT_TABLE table=default.Table_40c9009b608449f5ad959e33a2fbbc12 [id=ffb188d244694ce2aa7985340abb8813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.710464 4193 tablet_service.cc:1511] Processing CreateTablet for tablet 958ba8046c294be3ac20e2d93925d17b (DEFAULT_TABLE table=default.Table_40c9009b608449f5ad959e33a2fbbc12 [id=ffb188d244694ce2aa7985340abb8813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.710837 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958ba8046c294be3ac20e2d93925d17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.711488 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958ba8046c294be3ac20e2d93925d17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.715873 4776 tablet_bootstrap.cc:492] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:49.716244 4776 ts_tablet_manager.cc:1403] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.283s user 0.085s sys 0.132s
I20260430 19:58:49.721454 4470 leader_election.cc:304] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:49.723110 4911 raft_consensus.cc:2804] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:49.723635 4911 raft_consensus.cc:697] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:49.718462 4776 raft_consensus.cc:359] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.724316 4776 raft_consensus.cc:385] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.724586 4776 raft_consensus.cc:740] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.724464 4502 tablet_service.cc:1511] Processing CreateTablet for tablet ba4cf90db44540f2b438966361af8192 (DEFAULT_TABLE table=default.Table_c4c6ff375216499ba02c2f6fb6ad2db5 [id=254e27afbb6344d8ae93ae557ad802df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.724634 4911 consensus_queue.cc:237] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.725576 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4cf90db44540f2b438966361af8192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.725247 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_79d9a94d02f94e8baa483828c8218407"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:49.726774 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79d9a94d02f94e8baa483828c8218407 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.726445 4776 consensus_queue.cc:260] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.732689 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_f01cb568b13b4e5ab73b36deda0357f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:49.734272 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f01cb568b13b4e5ab73b36deda0357f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.735461 4729 raft_consensus.cc:359] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.736131 4729 raft_consensus.cc:385] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.735875 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_c8f20997cfad46fea29e3a23559df569"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:49.736366 4729 raft_consensus.cc:740] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
W20260430 19:58:49.737218 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8f20997cfad46fea29e3a23559df569 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.736937 4729 consensus_queue.cc:260] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.739274 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 70) took 2018 ms. Trace:
I20260430 19:58:49.740124 4742 rpcz_store.cc:276] 0430 19:58:47.720719 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.720851 (+ 132us) service_pool.cc:225] Handling call
0430 19:58:49.225142 (+1504291us) ts_tablet_manager.cc:1151] Deleting tablet 0e64ef047d8c46e9ac8387feb0dbe9ba
0430 19:58:49.225186 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.563882 (+338696us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.704006 (+140124us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.739206 (+ 35200us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1459846}
I20260430 19:58:49.751361 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 0e64ef047d8c46e9ac8387feb0dbe9ba (table default.Table_ccf1123d6d5d4e489838f28c6125aa15 [id=007139834a09457dbfcb9a426839d894]) successfully deleted
I20260430 19:58:49.752807 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 68) took 2035 ms. Trace:
I20260430 19:58:49.744784 4776 ts_tablet_manager.cc:1434] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260430 19:58:49.756021 4776 tablet_bootstrap.cc:492] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:49.756313 4742 rpcz_store.cc:276] 0430 19:58:47.717389 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:47.746723 (+ 29334us) service_pool.cc:225] Handling call
0430 19:58:49.752464 (+2005741us) ts_tablet_manager.cc:1151] Deleting tablet 6220177b3e3e4d18bd8e20e6d9d96be7
0430 19:58:49.752508 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.752752 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1902378}
W20260430 19:58:49.758139 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6220177b3e3e4d18bd8e20e6d9d96be7 already in progress: creating tablet
I20260430 19:58:49.759649 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 78) took 1726 ms. Trace:
I20260430 19:58:49.760486 4729 ts_tablet_manager.cc:1434] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.051s user 0.005s sys 0.002s
I20260430 19:58:49.762347 4192 tablet_service.cc:1511] Processing CreateTablet for tablet ba4cf90db44540f2b438966361af8192 (DEFAULT_TABLE table=default.Table_c4c6ff375216499ba02c2f6fb6ad2db5 [id=254e27afbb6344d8ae93ae557ad802df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.768350 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4cf90db44540f2b438966361af8192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.769253 4776 tablet_bootstrap.cc:654] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.777740 4358 tablet_service.cc:1511] Processing CreateTablet for tablet ba4cf90db44540f2b438966361af8192 (DEFAULT_TABLE table=default.Table_c4c6ff375216499ba02c2f6fb6ad2db5 [id=254e27afbb6344d8ae93ae557ad802df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.778884 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4cf90db44540f2b438966361af8192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.779881 4915 ts_tablet_manager.cc:1929] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:49.780614 4915 log.cc:1199] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/a7b625afadd744dc86d7bfe1da77876d
I20260430 19:58:49.785296 4742 rpcz_store.cc:276] 0430 19:58:48.034604 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.034741 (+ 137us) service_pool.cc:225] Handling call
0430 19:58:49.759206 (+1724465us) ts_tablet_manager.cc:1151] Deleting tablet e4e87fda401047cfa7070ae95b1701a9
0430 19:58:49.759251 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.759581 (+ 330us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1723910}
I20260430 19:58:49.786687 4729 tablet_bootstrap.cc:492] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:49.789955 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 80) took 1608 ms. Trace:
W20260430 19:58:49.791091 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet e4e87fda401047cfa7070ae95b1701a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4e87fda401047cfa7070ae95b1701a9 already in progress: creating tablet
I20260430 19:58:49.793097 4654 ts_tablet_manager.cc:1929] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:49.793617 4742 rpcz_store.cc:276] 0430 19:58:48.181590 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.181742 (+ 152us) service_pool.cc:225] Handling call
0430 19:58:49.789638 (+1607896us) ts_tablet_manager.cc:1151] Deleting tablet 13e8206ac4484abfa769276c83fd3333
0430 19:58:49.789679 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.789904 (+ 225us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1607286}
I20260430 19:58:49.799820 4900 raft_consensus.cc:493] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.802942 4900 raft_consensus.cc:515] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.804919 4900 leader_election.cc:290] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:49.807184 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 87) took 1296 ms. Trace:
I20260430 19:58:49.808135 4654 log.cc:1199] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/76073435ab7949a0bb0606e6efaa48da
I20260430 19:58:49.808305 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:49.808812 4742 rpcz_store.cc:276] 0430 19:58:48.511245 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.514720 (+ 3475us) service_pool.cc:225] Handling call
0430 19:58:49.806854 (+1292134us) ts_tablet_manager.cc:1151] Deleting tablet 395f97fdf3834cb9aaabd261e1bf76c5
0430 19:58:49.806895 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.807131 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1291644}
W20260430 19:58:49.809199 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 13e8206ac4484abfa769276c83fd3333 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13e8206ac4484abfa769276c83fd3333 already in progress: creating tablet
I20260430 19:58:49.809788 4742 tablet_replica.cc:333] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
W20260430 19:58:49.809875 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 395f97fdf3834cb9aaabd261e1bf76c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 395f97fdf3834cb9aaabd261e1bf76c5 already in progress: creating tablet
W20260430 19:58:49.810005 4158 leader_election.cc:343] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.810428 4742 raft_consensus.cc:2243] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:49.808454 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:49.811360 4654 ts_tablet_manager.cc:1950] T 76073435ab7949a0bb0606e6efaa48da P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:49.813227 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 97) took 1229 ms. Trace:
I20260430 19:58:49.813524 4654 rpcz_store.cc:276] 0430 19:58:48.583364 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.583511 (+ 147us) service_pool.cc:225] Handling call
0430 19:58:49.539170 (+955659us) ts_tablet_manager.cc:1151] Deleting tablet 76073435ab7949a0bb0606e6efaa48da
0430 19:58:49.539211 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.717753 (+178542us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.792972 (+ 75219us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.813169 (+ 20197us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":955116}
W20260430 19:58:49.811717 4161 leader_election.cc:343] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.814834 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]) successfully deleted
I20260430 19:58:49.814992 4915 ts_tablet_manager.cc:1950] T a7b625afadd744dc86d7bfe1da77876d P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:49.814930 4161 leader_election.cc:304] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:49.816084 4900 raft_consensus.cc:2749] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:49.817873 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet a7b625afadd744dc86d7bfe1da77876d (table default.Table_e0abcc152ab14699b99ab62f4bf1b0f8 [id=a0f9ecd533e74e449ffe30b7e4f274b0]) successfully deleted
I20260430 19:58:49.820150 4742 raft_consensus.cc:2272] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.821943 4742 ts_tablet_manager.cc:1916] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:49.825202 4818 tablet_bootstrap.cc:492] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:49.825798 4818 ts_tablet_manager.cc:1403] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.544s user 0.008s sys 0.013s
I20260430 19:58:49.825827 4915 tablet_replica.cc:333] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:49.826112 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 28ba03500c0540cda44debe362278e47 (DEFAULT_TABLE table=default.Table_308a505e52094e64933fe14eee977c7b [id=e2a73d32712842c783cbc616484340bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.827400 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ba03500c0540cda44debe362278e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.828068 4729 tablet_bootstrap.cc:654] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.828217 4818 raft_consensus.cc:359] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.828791 4818 raft_consensus.cc:385] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.829210 4818 raft_consensus.cc:740] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.829836 4818 consensus_queue.cc:260] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:49.830749 4654 tablet_replica.cc:333] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:49.835134 4654 raft_consensus.cc:2243] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:49.839051 4654 pending_rounds.cc:70] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:49.839180 4769 raft_consensus.cc:493] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.839362 4654 pending_rounds.cc:77] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964113432104960 op_type: NO_OP noop_request { }
I20260430 19:58:49.839675 4654 raft_consensus.cc:2889] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:49.839607 4769 raft_consensus.cc:515] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.839965 4654 raft_consensus.cc:2272] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.832883 4818 ts_tablet_manager.cc:1434] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260430 19:58:49.841064 4818 tablet_bootstrap.cc:492] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:49.841454 4769 leader_election.cc:290] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:49.843621 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6220177b3e3e4d18bd8e20e6d9d96be7" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:49.845120 4470 leader_election.cc:343] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.845185 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6220177b3e3e4d18bd8e20e6d9d96be7" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:49.846326 4818 tablet_bootstrap.cc:654] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:49.846580 4470 leader_election.cc:343] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.846963 4470 leader_election.cc:304] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:49.847993 4769 raft_consensus.cc:2749] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:49.848531 4654 ts_tablet_manager.cc:1916] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:49.850836 4027 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet e36490392a404b6bba29bd6803274b36
I20260430 19:58:49.851341 4915 raft_consensus.cc:2243] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:49.851845 4915 raft_consensus.cc:2272] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.852136 4027 catalog_manager.cc:6741] ReplaceTablet: tablet e36490392a404b6bba29bd6803274b36 deleted and replaced by tablet 34a6d662493e428aa475979abbffc9d7
I20260430 19:58:49.853008 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet e36490392a404b6bba29bd6803274b36 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a6d662493e428aa475979abbffc9d7) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:49.853642 4915 ts_tablet_manager.cc:1916] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:49.854246 4029 catalog_manager.cc:5671] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2). New cstate: current_term: 1 leader_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:49.860095 4192 tablet_service.cc:1558] Processing DeleteTablet for tablet e36490392a404b6bba29bd6803274b36 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a6d662493e428aa475979abbffc9d7) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:49.862911 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet e36490392a404b6bba29bd6803274b36 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a6d662493e428aa475979abbffc9d7) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:49.863343 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 28ba03500c0540cda44debe362278e47 (DEFAULT_TABLE table=default.Table_308a505e52094e64933fe14eee977c7b [id=e2a73d32712842c783cbc616484340bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.864408 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ba03500c0540cda44debe362278e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.871214 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 28ba03500c0540cda44debe362278e47 (DEFAULT_TABLE table=default.Table_308a505e52094e64933fe14eee977c7b [id=e2a73d32712842c783cbc616484340bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.872327 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ba03500c0540cda44debe362278e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.872404 4729 tablet_bootstrap.cc:492] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:49.872783 4729 ts_tablet_manager.cc:1403] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.086s user 0.010s sys 0.000s
I20260430 19:58:49.875952 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 28ba03500c0540cda44debe362278e47
I20260430 19:58:49.877919 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ba03500c0540cda44debe362278e47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:49.879308 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ba03500c0540cda44debe362278e47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:49.882830 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ba03500c0540cda44debe362278e47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:49.886065 4635 webserver.cc:492] Webserver started at http://127.3.190.62:38679/ using document root <none> and password file <none>
I20260430 19:58:49.885211 4505 tablet_service.cc:1511] Processing CreateTablet for tablet f5c54c37bdae458d88594a3f5c401a48 (DEFAULT_TABLE table=default.Table_5a90ad7e91c44b4ea6f4240ebe7e7d65 [id=3f9fb78a9e1b4b8a89881283e32e3c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.887207 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c54c37bdae458d88594a3f5c401a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.877465 4729 raft_consensus.cc:359] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.894985 4729 raft_consensus.cc:385] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.895306 4729 raft_consensus.cc:740] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.895954 4729 consensus_queue.cc:260] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.899518 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 37d6c54ae89946158cad45e741c1c0f0
I20260430 19:58:49.900921 4025 catalog_manager.cc:6741] ReplaceTablet: tablet 37d6c54ae89946158cad45e741c1c0f0 deleted and replaced by tablet bdf98515d8284e3b9b84480807e8108a
I20260430 19:58:49.901150 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet bdf98515d8284e3b9b84480807e8108a) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:49.903653 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet bdf98515d8284e3b9b84480807e8108a) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:49.904091 4729 ts_tablet_manager.cc:1434] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.031s user 0.006s sys 0.001s
I20260430 19:58:49.904488 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet bdf98515d8284e3b9b84480807e8108a) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:49.905026 4729 tablet_bootstrap.cc:492] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:49.906867 4026 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_fea5239c878e444d9865930366db22aa" } new_table_name: "default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3" modify_external_catalogs: true
I20260430 19:58:49.910871 4025 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_95bf05b38f054143a0f537b967e00b93" } new_table_name: "default.Table_d1543ae16be840548845a7781aa09c80" modify_external_catalogs: true
I20260430 19:58:49.912288 4818 tablet_bootstrap.cc:492] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:49.912727 4818 ts_tablet_manager.cc:1403] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.072s user 0.008s sys 0.000s
I20260430 19:58:49.915208 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ed5a601ae75142dc9cc64766e464b377"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:49.915926 4742 ts_tablet_manager.cc:1929] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
W20260430 19:58:49.916910 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed5a601ae75142dc9cc64766e464b377 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:49.917093 4818 raft_consensus.cc:359] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.917688 4818 raft_consensus.cc:385] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.917920 4818 raft_consensus.cc:740] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.918830 4635 fs_manager.cc:362] Metadata directory not provided
I20260430 19:58:49.919168 4635 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:58:49.920126 4742 log.cc:1199] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/76073435ab7949a0bb0606e6efaa48da
I20260430 19:58:49.920845 4742 ts_tablet_manager.cc:1950] T 76073435ab7949a0bb0606e6efaa48da P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:49.924045 4915 ts_tablet_manager.cc:1929] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:49.925011 4818 consensus_queue.cc:260] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.929167 4915 log.cc:1199] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/cfb8e475e89f434f98482fee1b45dbca
I20260430 19:58:49.931692 4915 ts_tablet_manager.cc:1950] T cfb8e475e89f434f98482fee1b45dbca P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:49.933409 4654 ts_tablet_manager.cc:1929] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:49.933775 4654 log.cc:1199] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/a7b625afadd744dc86d7bfe1da77876d
I20260430 19:58:49.940634 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet cfb8e475e89f434f98482fee1b45dbca (table default.Table_d13179c843c54969b98111d67f829f5f [id=37e26a1d9e4f40cc8fb14b9934ba3780]) successfully deleted
I20260430 19:58:49.942835 4915 tablet_replica.cc:333] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:49.943089 4818 ts_tablet_manager.cc:1434] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.030s user 0.007s sys 0.000s
I20260430 19:58:49.944705 4915 raft_consensus.cc:2243] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:49.945763 4205 tablet_service.cc:1511] Processing CreateTablet for tablet f5c54c37bdae458d88594a3f5c401a48 (DEFAULT_TABLE table=default.Table_5a90ad7e91c44b4ea6f4240ebe7e7d65 [id=3f9fb78a9e1b4b8a89881283e32e3c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.946110 4818 tablet_bootstrap.cc:492] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:49.946182 4915 raft_consensus.cc:2272] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.946810 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c54c37bdae458d88594a3f5c401a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.924527 4729 tablet_bootstrap.cc:654] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.934438 4654 ts_tablet_manager.cc:1950] T a7b625afadd744dc86d7bfe1da77876d P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:49.953233 4900 raft_consensus.cc:493] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:49.953707 4900 raft_consensus.cc:515] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.955535 4900 leader_election.cc:290] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:49.958035 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c119dd9a5d944eabd588f62a76da98e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:49.959566 4158 leader_election.cc:343] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.960343 4917 tablet.cc:1724] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:49.956760 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c119dd9a5d944eabd588f62a76da98e" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:49.961828 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 98) took 1360 ms. Trace:
W20260430 19:58:49.961920 4161 leader_election.cc:343] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:49.962321 4161 leader_election.cc:304] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:49.962159 4654 rpcz_store.cc:276] 0430 19:58:48.601928 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.732778 (+130850us) service_pool.cc:225] Handling call
0430 19:58:49.814547 (+1081769us) ts_tablet_manager.cc:1151] Deleting tablet a7b625afadd744dc86d7bfe1da77876d
0430 19:58:49.814583 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.891918 (+ 77335us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.933300 (+ 41382us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.961773 (+ 28473us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1081164}
I20260430 19:58:49.963040 4900 raft_consensus.cc:2749] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:49.969601 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet a7b625afadd744dc86d7bfe1da77876d (table default.Table_e0abcc152ab14699b99ab62f4bf1b0f8 [id=a0f9ecd533e74e449ffe30b7e4f274b0]) successfully deleted
I20260430 19:58:49.969856 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 100) took 1106 ms. Trace:
I20260430 19:58:49.971696 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 91) took 1365 ms. Trace:
I20260430 19:58:49.972002 4742 rpcz_store.cc:276] 0430 19:58:48.606024 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.609588 (+ 3564us) service_pool.cc:225] Handling call
0430 19:58:49.809644 (+1200056us) ts_tablet_manager.cc:1151] Deleting tablet 76073435ab7949a0bb0606e6efaa48da
0430 19:58:49.809684 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.862138 (+ 52454us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.915810 (+ 53672us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:49.971642 (+ 55832us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45749,"tablet-delete.queue_time_us":1167588}
I20260430 19:58:49.973531 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 92) took 1182 ms. Trace:
I20260430 19:58:49.973825 4742 rpcz_store.cc:276] 0430 19:58:48.790704 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.801237 (+ 10533us) service_pool.cc:225] Handling call
0430 19:58:49.973123 (+1171886us) ts_tablet_manager.cc:1151] Deleting tablet b6acec54bc7b4692894333e0150a1aff
0430 19:58:49.973158 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.973480 (+ 322us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1171221}
I20260430 19:58:49.974231 4818 tablet_bootstrap.cc:654] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:49.975170 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 93) took 1131 ms. Trace:
I20260430 19:58:49.975436 4742 rpcz_store.cc:276] 0430 19:58:48.844092 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.844197 (+ 105us) service_pool.cc:225] Handling call
0430 19:58:49.974844 (+1130647us) ts_tablet_manager.cc:1151] Deleting tablet 09f272340cd942e9b2af9e38a2bdec82
0430 19:58:49.974879 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.975122 (+ 243us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1130043}
I20260430 19:58:49.967350 4776 tablet_bootstrap.cc:492] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:49.976404 4776 ts_tablet_manager.cc:1403] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.221s user 0.032s sys 0.009s
I20260430 19:58:49.977140 4502 tablet_service.cc:1511] Processing CreateTablet for tablet ce21ac617c5b4d6ca97c6f68f53c6a67 (DEFAULT_TABLE table=default.Table_0c33415d44d44344a5740dc6cdb7da0c [id=d65b3e0c5edb4dfc81e0c9c4e9efbc7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.977401 4742 tablet_replica.cc:333] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:49.978132 4742 raft_consensus.cc:2243] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:49.978523 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce21ac617c5b4d6ca97c6f68f53c6a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.979017 4742 raft_consensus.cc:2272] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:49.979243 4776 raft_consensus.cc:359] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.979920 4729 tablet_bootstrap.cc:492] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:49.979885 4776 raft_consensus.cc:385] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.980136 4776 raft_consensus.cc:740] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.980304 4729 ts_tablet_manager.cc:1403] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.075s user 0.008s sys 0.000s
I20260430 19:58:49.980831 4776 consensus_queue.cc:260] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.981038 4358 tablet_service.cc:1511] Processing CreateTablet for tablet f5c54c37bdae458d88594a3f5c401a48 (DEFAULT_TABLE table=default.Table_5a90ad7e91c44b4ea6f4240ebe7e7d65 [id=3f9fb78a9e1b4b8a89881283e32e3c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.981923 4203 tablet_service.cc:1511] Processing CreateTablet for tablet ce21ac617c5b4d6ca97c6f68f53c6a67 (DEFAULT_TABLE table=default.Table_0c33415d44d44344a5740dc6cdb7da0c [id=d65b3e0c5edb4dfc81e0c9c4e9efbc7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:49.970175 4654 rpcz_store.cc:276] 0430 19:58:48.863182 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.863319 (+ 137us) service_pool.cc:225] Handling call
0430 19:58:49.969517 (+1106198us) ts_tablet_manager.cc:1151] Deleting tablet 09f272340cd942e9b2af9e38a2bdec82
0430 19:58:49.969567 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.969800 (+ 233us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1105257}
I20260430 19:58:49.982928 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5c54c37bdae458d88594a3f5c401a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.983058 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce21ac617c5b4d6ca97c6f68f53c6a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:49.983606 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 99) took 1172 ms. Trace:
I20260430 19:58:49.983899 4654 rpcz_store.cc:276] 0430 19:58:48.810773 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.870068 (+ 59295us) service_pool.cc:225] Handling call
0430 19:58:49.983275 (+1113207us) ts_tablet_manager.cc:1151] Deleting tablet b6acec54bc7b4692894333e0150a1aff
0430 19:58:49.983315 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.983545 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1110437}
I20260430 19:58:49.984805 4729 raft_consensus.cc:359] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.985390 4729 raft_consensus.cc:385] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:49.985625 4729 raft_consensus.cc:740] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:49.991192 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 76073435ab7949a0bb0606e6efaa48da (table default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]) successfully deleted
W20260430 19:58:49.991880 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet b6acec54bc7b4692894333e0150a1aff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6acec54bc7b4692894333e0150a1aff already in progress: creating tablet
W20260430 19:58:49.992478 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 09f272340cd942e9b2af9e38a2bdec82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09f272340cd942e9b2af9e38a2bdec82 already in progress: creating tablet
I20260430 19:58:49.992344 4361 tablet_service.cc:1511] Processing CreateTablet for tablet ce21ac617c5b4d6ca97c6f68f53c6a67 (DEFAULT_TABLE table=default.Table_0c33415d44d44344a5740dc6cdb7da0c [id=d65b3e0c5edb4dfc81e0c9c4e9efbc7a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:49.993032 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 0e62335e43c1410b8b3acdd9e2f47981 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e62335e43c1410b8b3acdd9e2f47981 already in progress: creating tablet
I20260430 19:58:49.993263 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce21ac617c5b4d6ca97c6f68f53c6a67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:49.993639 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 09f272340cd942e9b2af9e38a2bdec82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09f272340cd942e9b2af9e38a2bdec82 already in progress: creating tablet
W20260430 19:58:49.994141 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet b6acec54bc7b4692894333e0150a1aff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6acec54bc7b4692894333e0150a1aff already in progress: creating tablet
I20260430 19:58:49.994868 4729 consensus_queue.cc:260] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:49.996158 4654 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 101) took 1007 ms. Trace:
I20260430 19:58:49.996447 4654 rpcz_store.cc:276] 0430 19:58:48.989104 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.989249 (+ 145us) service_pool.cc:225] Handling call
0430 19:58:49.995851 (+1006602us) ts_tablet_manager.cc:1151] Deleting tablet 0e62335e43c1410b8b3acdd9e2f47981
0430 19:58:49.995892 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:49.996107 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1005182}
I20260430 19:58:50.003051 4654 tablet_replica.cc:333] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:50.003727 4654 raft_consensus.cc:2243] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.004241 4654 raft_consensus.cc:2272] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.004587 4776 ts_tablet_manager.cc:1434] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.028s user 0.004s sys 0.002s
I20260430 19:58:50.005344 4729 ts_tablet_manager.cc:1434] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.025s user 0.006s sys 0.001s
I20260430 19:58:50.007532 4729 tablet_bootstrap.cc:492] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:50.014143 4776 tablet_bootstrap.cc:492] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
W20260430 19:58:50.014748 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 0e62335e43c1410b8b3acdd9e2f47981 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e62335e43c1410b8b3acdd9e2f47981 already in progress: creating tablet
I20260430 19:58:50.019765 4776 tablet_bootstrap.cc:654] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.031388 4818 tablet_bootstrap.cc:492] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:50.031345 4654 ts_tablet_manager.cc:1916] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.031836 4818 ts_tablet_manager.cc:1403] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.086s user 0.010s sys 0.000s
I20260430 19:58:50.039045 4769 raft_consensus.cc:493] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.039489 4769 raft_consensus.cc:515] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.040539 4361 tablet_service.cc:1511] Processing CreateTablet for tablet d1f7489f1d9c46b8b33745c3e03a2138 (DEFAULT_TABLE table=default.Table_abcdd9deb61842beb234605de671f749 [id=9f9849e27c66414e8544c61587c01623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:50.041280 4769 leader_election.cc:290] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.041576 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1f7489f1d9c46b8b33745c3e03a2138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:50.042708 4196 tablet_service.cc:1511] Processing CreateTablet for tablet d1f7489f1d9c46b8b33745c3e03a2138 (DEFAULT_TABLE table=default.Table_abcdd9deb61842beb234605de671f749 [id=9f9849e27c66414e8544c61587c01623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:50.043365 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e34f3b1bf84f72855cf87683c5e318" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:50.043836 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1f7489f1d9c46b8b33745c3e03a2138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:50.044988 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e34f3b1bf84f72855cf87683c5e318" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:50.045590 4382 raft_consensus.cc:2468] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:50.047199 4470 leader_election.cc:304] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
W20260430 19:58:50.048038 4470 leader_election.cc:343] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.048959 4729 tablet_bootstrap.cc:654] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.049639 4769 raft_consensus.cc:2804] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:50.049995 4769 raft_consensus.cc:493] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:50.050369 4769 raft_consensus.cc:3060] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.034185 4818 raft_consensus.cc:359] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.051884 4818 raft_consensus.cc:385] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.052160 4818 raft_consensus.cc:740] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
W20260430 19:58:50.052498 4015 consensus_peers.cc:597] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 -> Peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Couldn't send request to peer 6b0a440dabed4814aaefe431944a7b5e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260430 19:58:50.052858 4818 consensus_queue.cc:260] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.053606 4742 ts_tablet_manager.cc:1916] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.057072 4818 ts_tablet_manager.cc:1434] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260430 19:58:50.038408 4505 tablet_service.cc:1511] Processing CreateTablet for tablet d1f7489f1d9c46b8b33745c3e03a2138 (DEFAULT_TABLE table=default.Table_abcdd9deb61842beb234605de671f749 [id=9f9849e27c66414e8544c61587c01623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:50.066648 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1f7489f1d9c46b8b33745c3e03a2138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:50.067544 4776 tablet_bootstrap.cc:492] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:50.074317 4776 ts_tablet_manager.cc:1403] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.060s user 0.009s sys 0.003s
I20260430 19:58:50.076804 4776 raft_consensus.cc:359] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.077500 4776 raft_consensus.cc:385] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.077749 4776 raft_consensus.cc:740] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.078281 4776 consensus_queue.cc:260] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.080361 4776 ts_tablet_manager.cc:1434] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260430 19:58:50.081128 4776 tablet_bootstrap.cc:492] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:50.080986 4022 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:50.081666 4022 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 0832f9191b7f4c3683f37cce8966b266 (table default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]). Expected version 1 got 0
W20260430 19:58:50.085564 4026 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6cef0197c00b4ab7a11e2e5123a4c277 consensus state has no leader
I20260430 19:58:50.090641 4021 catalog_manager.cc:5671] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:50.092583 4021 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:50.093807 4023 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet f3db0439f4f042a4a9cb38ebd3764ab5
I20260430 19:58:50.094098 4029 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cf6f9cf5be37436f9671fa67edc046a1" } modify_external_catalogs: true
I20260430 19:58:50.095448 4029 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cf6f9cf5be37436f9671fa67edc046a1" } modify_external_catalogs: true
I20260430 19:58:50.106417 4818 tablet_bootstrap.cc:492] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:50.112360 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet a7b625afadd744dc86d7bfe1da77876d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.112982 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet f3db0439f4f042a4a9cb38ebd3764ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.115453 4192 tablet_service.cc:1558] Processing DeleteTablet for tablet f3db0439f4f042a4a9cb38ebd3764ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.116178 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet a7b625afadd744dc86d7bfe1da77876d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:46 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.125352 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet f3db0439f4f042a4a9cb38ebd3764ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:50.127513 4776 tablet_bootstrap.cc:654] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.135535 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 3452fa52c20142f7bf396bc182a829a7 (DEFAULT_TABLE table=default.Table_9d21910e9cb14cc6a75a5432b143c915 [id=69dac89c4d4a4081aed4ac000a4a51c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:50.136615 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3452fa52c20142f7bf396bc182a829a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:50.140077 4026 master_service.cc:954] ReplaceTablet: received request to replace tablet 70d410f3b3b04e649b79dda04c0891ee from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:50.065266 4769 raft_consensus.cc:515] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.146533 4769 leader_election.cc:290] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.146469 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e34f3b1bf84f72855cf87683c5e318" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:50.147044 4382 raft_consensus.cc:3060] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.150358 4654 ts_tablet_manager.cc:1929] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:50.150821 4654 log.cc:1199] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/e36490392a404b6bba29bd6803274b36
I20260430 19:58:50.151610 4654 ts_tablet_manager.cc:1950] T e36490392a404b6bba29bd6803274b36 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:50.153733 4382 raft_consensus.cc:2468] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:50.154862 4470 leader_election.cc:304] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:50.155594 4911 raft_consensus.cc:493] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.155984 4911 raft_consensus.cc:515] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.156880 4742 ts_tablet_manager.cc:1929] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:50.157222 4742 log.cc:1199] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/e36490392a404b6bba29bd6803274b36
I20260430 19:58:50.157774 4911 leader_election.cc:290] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.158196 4911 raft_consensus.cc:2804] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:50.158527 4742 ts_tablet_manager.cc:1950] T e36490392a404b6bba29bd6803274b36 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:50.158582 4911 raft_consensus.cc:697] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:50.160990 4911 consensus_queue.cc:237] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.160467 4742 tablet_replica.cc:333] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:50.163280 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3db0439f4f042a4a9cb38ebd3764ab5" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:50.164014 4742 raft_consensus.cc:2243] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.164427 4742 raft_consensus.cc:2272] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.165762 4635 fs_manager.cc:714] Time spent opening directory manager: real 0.178s user 0.006s sys 0.001s
W20260430 19:58:50.166206 4470 leader_election.cc:343] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.170244 4729 tablet_bootstrap.cc:492] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:50.170691 4729 ts_tablet_manager.cc:1403] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.163s user 0.005s sys 0.005s
I20260430 19:58:50.178099 4935 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:58:50.179111 4729 raft_consensus.cc:359] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.179548 4635 fs_manager.cc:730] Time spent opening block manager: real 0.010s user 0.005s sys 0.000s
I20260430 19:58:50.179737 4729 raft_consensus.cc:385] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.179991 4729 raft_consensus.cc:740] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.180536 4729 consensus_queue.cc:260] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.182440 4729 ts_tablet_manager.cc:1434] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260430 19:58:50.183367 4729 tablet_bootstrap.cc:492] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:50.179993 4635 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
uuid: "6b0a440dabed4814aaefe431944a7b5e"
format_stamp: "Formatted at 2026-04-30 19:58:27 on dist-test-slave-6zcr"
I20260430 19:58:50.183952 4635 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:58:50.186328 4769 raft_consensus.cc:493] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.186769 4769 raft_consensus.cc:515] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.188552 4769 leader_election.cc:290] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.192188 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e8206ac4484abfa769276c83fd3333" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:50.192818 4382 raft_consensus.cc:2468] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:50.193874 4915 ts_tablet_manager.cc:1916] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.194388 4470 leader_election.cc:304] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:50.195150 4769 raft_consensus.cc:2804] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:50.195441 4769 raft_consensus.cc:493] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:50.195835 4769 raft_consensus.cc:3060] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.201910 4769 raft_consensus.cc:515] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.202656 4742 ts_tablet_manager.cc:1916] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.203357 4776 tablet_bootstrap.cc:492] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:50.203758 4776 ts_tablet_manager.cc:1403] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.123s user 0.015s sys 0.007s
I20260430 19:58:50.205020 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet e36490392a404b6bba29bd6803274b36 (table default.Table_028cf8f724c64a51adefa0cc9c15a50e [id=cd706c6a34ac49a4b82e411aca27698a]) successfully deleted
W20260430 19:58:50.205747 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 28ba03500c0540cda44debe362278e47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ba03500c0540cda44debe362278e47 already in progress: creating tablet
I20260430 19:58:50.206029 4776 raft_consensus.cc:359] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:50.206416 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 37d6c54ae89946158cad45e741c1c0f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37d6c54ae89946158cad45e741c1c0f0 already in progress: creating tablet
I20260430 19:58:50.206295 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 3452fa52c20142f7bf396bc182a829a7 (DEFAULT_TABLE table=default.Table_9d21910e9cb14cc6a75a5432b143c915 [id=69dac89c4d4a4081aed4ac000a4a51c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:50.206655 4776 raft_consensus.cc:385] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.206884 4776 raft_consensus.cc:740] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.207441 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3452fa52c20142f7bf396bc182a829a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:50.207361 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 3452fa52c20142f7bf396bc182a829a7 (DEFAULT_TABLE table=default.Table_9d21910e9cb14cc6a75a5432b143c915 [id=69dac89c4d4a4081aed4ac000a4a51c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:50.207645 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet e36490392a404b6bba29bd6803274b36 (table default.Table_028cf8f724c64a51adefa0cc9c15a50e [id=cd706c6a34ac49a4b82e411aca27698a]) successfully deleted
I20260430 19:58:50.207425 4776 consensus_queue.cc:260] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:50.208174 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 28ba03500c0540cda44debe362278e47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ba03500c0540cda44debe362278e47 already in progress: creating tablet
I20260430 19:58:50.208420 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3452fa52c20142f7bf396bc182a829a7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:50.208714 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 37d6c54ae89946158cad45e741c1c0f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37d6c54ae89946158cad45e741c1c0f0 already in progress: creating tablet
W20260430 19:58:50.209239 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet a7b625afadd744dc86d7bfe1da77876d because the tablet was not found. No further retry: Not found: Tablet not found: a7b625afadd744dc86d7bfe1da77876d
W20260430 19:58:50.209767 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet f3db0439f4f042a4a9cb38ebd3764ab5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f3db0439f4f042a4a9cb38ebd3764ab5 already in progress: creating tablet
I20260430 19:58:50.219977 4776 ts_tablet_manager.cc:1434] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.016s user 0.005s sys 0.001s
I20260430 19:58:50.220916 4776 tablet_bootstrap.cc:492] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:50.220916 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e8206ac4484abfa769276c83fd3333" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:50.221459 4382 raft_consensus.cc:3060] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.221773 4915 ts_tablet_manager.cc:1929] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260430 19:58:50.222158 4915 log.cc:1199] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/e36490392a404b6bba29bd6803274b36
I20260430 19:58:50.222906 4915 ts_tablet_manager.cc:1950] T e36490392a404b6bba29bd6803274b36 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:50.228094 4729 tablet_bootstrap.cc:654] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.231133 4769 leader_election.cc:290] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.235041 4028 master_service.cc:954] ReplaceTablet: received request to replace tablet 13e8206ac4484abfa769276c83fd3333 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:50.235854 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_90a4740b66574861a91bc789cb56b353"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:50.236204 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e34f3b1bf84f72855cf87683c5e318" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
W20260430 19:58:50.237319 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90a4740b66574861a91bc789cb56b353 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:50.238307 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e8206ac4484abfa769276c83fd3333" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:50.238934 4222 raft_consensus.cc:2468] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:50.239084 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e8206ac4484abfa769276c83fd3333" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:50.239233 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_6e4f222e6ba54644a2e2da32762001f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:50.239595 4223 raft_consensus.cc:3060] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:50.240609 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e4f222e6ba54644a2e2da32762001f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:50.244874 4470 leader_election.cc:343] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.245967 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3db0439f4f042a4a9cb38ebd3764ab5" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:50.248461 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet e36490392a404b6bba29bd6803274b36 (table default.Table_028cf8f724c64a51adefa0cc9c15a50e [id=cd706c6a34ac49a4b82e411aca27698a]) successfully deleted
W20260430 19:58:50.253005 4470 leader_election.cc:343] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:50.253381 4470 leader_election.cc:304] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:50.255991 4776 tablet_bootstrap.cc:654] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.258069 4885 consensus_queue.cc:1048] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:58:50.260459 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 28ba03500c0540cda44debe362278e47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28ba03500c0540cda44debe362278e47 already in progress: creating tablet
I20260430 19:58:50.251664 4223 raft_consensus.cc:2468] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
W20260430 19:58:50.266863 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 37d6c54ae89946158cad45e741c1c0f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37d6c54ae89946158cad45e741c1c0f0 already in progress: creating tablet
I20260430 19:58:50.266886 4470 leader_election.cc:304] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:50.267571 4911 raft_consensus.cc:2749] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:50.268137 4911 raft_consensus.cc:2804] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:50.271373 4818 tablet_bootstrap.cc:700] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e: Time spent opening tablet: real 0.155s user 0.002s sys 0.000s
I20260430 19:58:50.268729 4911 raft_consensus.cc:697] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:50.272022 4818 tablet_bootstrap.cc:654] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.272325 4911 consensus_queue.cc:237] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.268347 4915 tablet_replica.cc:333] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:50.276144 4915 raft_consensus.cc:2243] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.276777 4915 raft_consensus.cc:2272] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.278425 4729 tablet_bootstrap.cc:492] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:50.278798 4729 ts_tablet_manager.cc:1403] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.096s user 0.014s sys 0.004s
I20260430 19:58:50.279930 4382 raft_consensus.cc:2468] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:50.281610 4729 raft_consensus.cc:359] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.282196 4729 raft_consensus.cc:385] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.282449 4729 raft_consensus.cc:740] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.287635 4729 consensus_queue.cc:260] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.291370 4729 ts_tablet_manager.cc:1434] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
W20260430 19:58:50.293576 4025 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cfd78a1cd7a94e27bb376d1b7c151073 has no consensus state
I20260430 19:58:50.293395 4729 tablet_bootstrap.cc:492] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:50.302726 4915 ts_tablet_manager.cc:1916] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.316840 4729 tablet_bootstrap.cc:654] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.319507 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet e36490392a404b6bba29bd6803274b36 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a6d662493e428aa475979abbffc9d7) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:50.327804 4776 tablet_bootstrap.cc:492] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:50.328229 4776 ts_tablet_manager.cc:1403] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.108s user 0.010s sys 0.018s
I20260430 19:58:50.330626 4776 raft_consensus.cc:359] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.331260 4776 raft_consensus.cc:385] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.331519 4776 raft_consensus.cc:740] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.332087 4776 consensus_queue.cc:260] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.340058 4915 ts_tablet_manager.cc:1929] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260430 19:58:50.342412 4016 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.61:35559, user_credentials={real_user=slave}} blocked reactor thread for 132007us
I20260430 19:58:50.344046 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet e36490392a404b6bba29bd6803274b36 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a6d662493e428aa475979abbffc9d7) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.344866 4196 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e8206ac4484abfa769276c83fd3333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.346735 4776 ts_tablet_manager.cc:1434] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20260430 19:58:50.352766 4776 tablet_bootstrap.cc:492] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:50.355932 4885 consensus_queue.cc:1048] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:50.362779 4915 log.cc:1199] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/f3db0439f4f042a4a9cb38ebd3764ab5
I20260430 19:58:50.363466 4915 ts_tablet_manager.cc:1950] T f3db0439f4f042a4a9cb38ebd3764ab5 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:50.364626 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:50.364995 4776 tablet_bootstrap.cc:654] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.366122 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet f3db0439f4f042a4a9cb38ebd3764ab5 (table default.Table_7e8d384dfa62442fbf585d952b02fe29 [id=977b7bf53cdf45158ea1fb1a4d2fdf05]) successfully deleted
W20260430 19:58:50.371840 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet e36490392a404b6bba29bd6803274b36 because the tablet was not found. No further retry: Not found: Tablet not found: e36490392a404b6bba29bd6803274b36
I20260430 19:58:50.372643 4915 tablet_replica.cc:333] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:50.373410 4915 raft_consensus.cc:2243] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.374300 4915 raft_consensus.cc:2272] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.343705 4742 ts_tablet_manager.cc:1929] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:50.379271 4742 log.cc:1199] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/f3db0439f4f042a4a9cb38ebd3764ab5
I20260430 19:58:50.379940 4742 ts_tablet_manager.cc:1950] T f3db0439f4f042a4a9cb38ebd3764ab5 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:50.380259 4776 tablet_bootstrap.cc:492] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:50.380667 4776 ts_tablet_manager.cc:1403] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.000s
I20260430 19:58:50.382138 4742 tablet_replica.cc:333] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:50.382969 4915 ts_tablet_manager.cc:1916] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.387741 4818 tablet_bootstrap.cc:492] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:50.388145 4818 ts_tablet_manager.cc:1403] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.282s user 0.009s sys 0.000s
I20260430 19:58:50.390623 4818 raft_consensus.cc:359] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.391227 4818 raft_consensus.cc:385] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.394793 4742 raft_consensus.cc:2243] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.395287 4742 raft_consensus.cc:2272] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.382969 4776 raft_consensus.cc:359] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.395800 4776 raft_consensus.cc:385] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.396076 4776 raft_consensus.cc:740] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.398041 4729 tablet_bootstrap.cc:492] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:50.398283 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet f3db0439f4f042a4a9cb38ebd3764ab5 (table default.Table_7e8d384dfa62442fbf585d952b02fe29 [id=977b7bf53cdf45158ea1fb1a4d2fdf05]) successfully deleted
I20260430 19:58:50.398439 4729 ts_tablet_manager.cc:1403] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.105s user 0.013s sys 0.011s
I20260430 19:58:50.400578 4729 raft_consensus.cc:359] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.401121 4729 raft_consensus.cc:385] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.401346 4729 raft_consensus.cc:740] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.401845 4729 consensus_queue.cc:260] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.402221 4742 ts_tablet_manager.cc:1916] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.410364 4729 ts_tablet_manager.cc:1434] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.012s user 0.004s sys 0.001s
I20260430 19:58:50.417768 4729 tablet_bootstrap.cc:492] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:50.419150 4942 raft_consensus.cc:493] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.419627 4942 raft_consensus.cc:515] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.421478 4942 leader_election.cc:290] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.422933 4729 tablet_bootstrap.cc:654] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.423316 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2ce1d43f214c8d9c1a1001baf53440" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:50.423863 4530 raft_consensus.cc:2468] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:50.425040 4317 leader_election.cc:304] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:50.396669 4776 consensus_queue.cc:260] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.425868 4942 raft_consensus.cc:2804] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:50.426159 4942 raft_consensus.cc:493] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:50.426431 4942 raft_consensus.cc:3060] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.431241 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_c4c6ff375216499ba02c2f6fb6ad2db5" } modify_external_catalogs: true
I20260430 19:58:50.433753 4776 ts_tablet_manager.cc:1434] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20260430 19:58:50.434082 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_c4c6ff375216499ba02c2f6fb6ad2db5" } modify_external_catalogs: true
I20260430 19:58:50.433449 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_40c9009b608449f5ad959e33a2fbbc12" } new_table_name: "default.Table_7b77d87732e142a7b0687c71ce62d0c3" modify_external_catalogs: true
I20260430 19:58:50.432278 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_e36d0112106b45e99f2ec17696ab9aac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:50.437119 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2ce1d43f214c8d9c1a1001baf53440" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:50.437682 4225 raft_consensus.cc:2468] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
W20260430 19:58:50.438284 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e36d0112106b45e99f2ec17696ab9aac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:50.439412 4915 ts_tablet_manager.cc:1929] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:50.439785 4915 log.cc:1199] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/6220177b3e3e4d18bd8e20e6d9d96be7
I20260430 19:58:50.440397 4915 ts_tablet_manager.cc:1950] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:50.440995 4776 tablet_bootstrap.cc:492] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:50.442745 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 6220177b3e3e4d18bd8e20e6d9d96be7 (table default.Table_0a03c960230b4206944b333b146c39c7 [id=26c468440dce47839c5c48e7bc76c173]) successfully deleted
I20260430 19:58:50.450227 4776 tablet_bootstrap.cc:654] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.434232 4942 raft_consensus.cc:515] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.456175 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2ce1d43f214c8d9c1a1001baf53440" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:50.456712 4530 raft_consensus.cc:3060] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.461637 4023 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
I20260430 19:58:50.464547 4029 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet ad2ce1d43f214c8d9c1a1001baf53440
I20260430 19:58:50.465044 4530 raft_consensus.cc:2468] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:50.466216 4317 leader_election.cc:304] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:50.467027 4947 raft_consensus.cc:2804] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:50.468012 4947 raft_consensus.cc:697] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:50.468688 4947 consensus_queue.cc:237] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.473650 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet ad2ce1d43f214c8d9c1a1001baf53440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:50.474462 4915 tablet_replica.cc:333] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:50.475262 4915 raft_consensus.cc:2243] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.475713 4915 raft_consensus.cc:2272] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.476135 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2ce1d43f214c8d9c1a1001baf53440" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:50.476656 4225 raft_consensus.cc:3060] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.478440 4915 ts_tablet_manager.cc:1916] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.483362 4225 raft_consensus.cc:2468] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:50.487581 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet bdf98515d8284e3b9b84480807e8108a) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.489883 4942 leader_election.cc:290] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.391474 4818 raft_consensus.cc:740] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.492738 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet ad2ce1d43f214c8d9c1a1001baf53440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.491303 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet ad2ce1d43f214c8d9c1a1001baf53440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.493656 4654 tablet_replica.cc:333] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:50.494210 4818 consensus_queue.cc:260] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.496373 4818 ts_tablet_manager.cc:1434] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.108s user 0.007s sys 0.000s
I20260430 19:58:50.497200 4654 raft_consensus.cc:2243] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:50.498291 4654 pending_rounds.cc:70] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:50.498485 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet e36490392a404b6bba29bd6803274b36 with delete_type TABLET_DATA_DELETED (replaced by tablet 34a6d662493e428aa475979abbffc9d7) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.499969 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e8206ac4484abfa769276c83fd3333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.498514 4654 pending_rounds.cc:77] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964118411517952 op_type: NO_OP noop_request { }
I20260430 19:58:50.501362 4654 raft_consensus.cc:2889] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:50.501681 4654 raft_consensus.cc:2272] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.519548 4635 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:58:50.521158 4900 raft_consensus.cc:493] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.521636 4900 raft_consensus.cc:515] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.523677 4900 leader_election.cc:290] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:50.525507 4915 ts_tablet_manager.cc:1929] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:50.525410 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb14746d8564bf8833e933a01771492" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:50.525835 4915 log.cc:1199] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/ad2ce1d43f214c8d9c1a1001baf53440
I20260430 19:58:50.526435 4915 ts_tablet_manager.cc:1950] T ad2ce1d43f214c8d9c1a1001baf53440 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
W20260430 19:58:50.526861 4161 leader_election.cc:343] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.528159 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb14746d8564bf8833e933a01771492" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:50.529400 4158 leader_election.cc:343] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.529815 4158 leader_election.cc:304] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:50.530829 4742 ts_tablet_manager.cc:1929] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:50.530906 4900 raft_consensus.cc:2749] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:50.534969 4769 consensus_queue.cc:1048] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:50.521514 4635 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:58:50.539050 4635 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260430 19:58:50.528409 4957 consensus_queue.cc:1175] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260430 19:58:50.545995 4654 ts_tablet_manager.cc:1916] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.531179 4742 log.cc:1199] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/a7b625afadd744dc86d7bfe1da77876d
I20260430 19:58:50.546942 4742 ts_tablet_manager.cc:1950] T a7b625afadd744dc86d7bfe1da77876d P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:50.497334 4818 tablet_bootstrap.cc:492] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:50.553815 4742 tablet_replica.cc:333] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:50.558182 4818 tablet_bootstrap.cc:654] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.560740 4742 raft_consensus.cc:2243] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.564026 4742 raft_consensus.cc:2272] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.570459 4742 ts_tablet_manager.cc:1916] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:50.576025 4014 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet cfd78a1cd7a94e27bb376d1b7c151073 (table default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cfd78a1cd7a94e27bb376d1b7c151073 has no consensus state
I20260430 19:58:50.576690 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet ad2ce1d43f214c8d9c1a1001baf53440 (table default.Table_cf6f9cf5be37436f9671fa67edc046a1 [id=043f0b9bc9274b9fbdbdd17fb0d6689c]) successfully deleted
I20260430 19:58:50.577455 4769 raft_consensus.cc:493] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.577907 4769 raft_consensus.cc:515] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.579680 4769 leader_election.cc:290] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.580339 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet e4e87fda401047cfa7070ae95b1701a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.581449 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:50.581782 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet a7b625afadd744dc86d7bfe1da77876d (table default.Table_e0abcc152ab14699b99ab62f4bf1b0f8 [id=a0f9ecd533e74e449ffe30b7e4f274b0]) successfully deleted
I20260430 19:58:50.583129 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:50.583604 4470 leader_election.cc:343] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.583716 4225 raft_consensus.cc:2468] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:50.584959 4654 ts_tablet_manager.cc:1929] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:50.584838 4470 leader_election.cc:304] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:50.585299 4654 log.cc:1199] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/ad2ce1d43f214c8d9c1a1001baf53440
I20260430 19:58:50.585969 4654 ts_tablet_manager.cc:1950] T ad2ce1d43f214c8d9c1a1001baf53440 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:50.587177 4911 raft_consensus.cc:2804] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:50.587534 4911 raft_consensus.cc:493] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:50.587831 4911 raft_consensus.cc:3060] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.588411 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet ad2ce1d43f214c8d9c1a1001baf53440 (table default.Table_cf6f9cf5be37436f9671fa67edc046a1 [id=043f0b9bc9274b9fbdbdd17fb0d6689c]) successfully deleted
I20260430 19:58:50.594794 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a3dc085505d24384b373f9edc7361950"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:50.596050 4382 raft_consensus.cc:1275] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Refusing update from remote peer 8a11a9a7f244435b8e84554e3fabbd0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260430 19:58:50.596904 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3dc085505d24384b373f9edc7361950 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:50.598047 4225 raft_consensus.cc:1275] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Refusing update from remote peer 8a11a9a7f244435b8e84554e3fabbd0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:58:50.602093 4654 tablet_replica.cc:333] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:50.603652 4654 raft_consensus.cc:2243] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.604089 4654 raft_consensus.cc:2272] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.605088 4912 consensus_queue.cc:1048] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:58:50.606645 4912 consensus_queue.cc:1048] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:50.606937 4654 ts_tablet_manager.cc:1916] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.598301 4029 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5bf270e5198e4ed491b5536ff5ed2629" } new_table_name: "default.Table_04b53e66a05c4462860e4411b8eb2312" modify_external_catalogs: true
W20260430 19:58:50.601142 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet e36490392a404b6bba29bd6803274b36 because the tablet was not found. No further retry: Not found: Tablet not found: e36490392a404b6bba29bd6803274b36
I20260430 19:58:50.590461 4023 catalog_manager.cc:5671] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } }
I20260430 19:58:50.622339 4023 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:50.625041 4911 raft_consensus.cc:515] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.632488 4769 consensus_queue.cc:1048] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:50.646111 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 70d410f3b3b04e649b79dda04c0891ee
I20260430 19:58:50.647567 4026 catalog_manager.cc:6741] ReplaceTablet: tablet 70d410f3b3b04e649b79dda04c0891ee deleted and replaced by tablet b0d0ac18ef9040c8a5bd804178081083
I20260430 19:58:50.649089 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 70d410f3b3b04e649b79dda04c0891ee with delete_type TABLET_DATA_DELETED (replaced by tablet b0d0ac18ef9040c8a5bd804178081083) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:50.650676 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 70d410f3b3b04e649b79dda04c0891ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 70d410f3b3b04e649b79dda04c0891ee already in progress: creating tablet
I20260430 19:58:50.654155 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 70d410f3b3b04e649b79dda04c0891ee with delete_type TABLET_DATA_DELETED (replaced by tablet b0d0ac18ef9040c8a5bd804178081083) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.655225 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 70d410f3b3b04e649b79dda04c0891ee with delete_type TABLET_DATA_DELETED (replaced by tablet b0d0ac18ef9040c8a5bd804178081083) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.660533 4729 tablet_bootstrap.cc:492] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:50.661115 4729 ts_tablet_manager.cc:1403] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.244s user 0.033s sys 0.025s
I20260430 19:58:50.663427 4729 raft_consensus.cc:359] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.664026 4729 raft_consensus.cc:385] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.670912 4776 tablet_bootstrap.cc:492] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:50.671329 4776 ts_tablet_manager.cc:1403] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.231s user 0.016s sys 0.030s
I20260430 19:58:50.672730 4020 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_abcdd9deb61842beb234605de671f749" } modify_external_catalogs: true
I20260430 19:58:50.673408 4020 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_abcdd9deb61842beb234605de671f749" } modify_external_catalogs: true
I20260430 19:58:50.664271 4729 raft_consensus.cc:740] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.681217 4729 consensus_queue.cc:260] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.683084 4729 ts_tablet_manager.cc:1434] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260430 19:58:50.693094 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
W20260430 19:58:50.691035 4027 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 958ba8046c294be3ac20e2d93925d17b consensus state has no leader
I20260430 19:58:50.712886 4976 tablet.cc:1724] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260430 19:58:50.718817 4911 leader_election.cc:290] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.724889 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet 2c119dd9a5d944eabd588f62a76da98e from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:50.729244 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:50.729758 4225 raft_consensus.cc:3060] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.733078 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet ba4cf90db44540f2b438966361af8192
I20260430 19:58:50.734500 4729 tablet_bootstrap.cc:492] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:50.735988 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet ba4cf90db44540f2b438966361af8192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:50.736284 4225 raft_consensus.cc:2468] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:50.723444 4971 tablet.cc:1724] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260430 19:58:50.738287 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet ba4cf90db44540f2b438966361af8192 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba4cf90db44540f2b438966361af8192 already in progress: creating tablet
W20260430 19:58:50.738788 4470 leader_election.cc:343] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.741128 4818 tablet_bootstrap.cc:492] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:50.741549 4818 ts_tablet_manager.cc:1403] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.244s user 0.008s sys 0.001s
I20260430 19:58:50.742204 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet ba4cf90db44540f2b438966361af8192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.743564 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet ba4cf90db44540f2b438966361af8192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.751955 4974 tablet.cc:1724] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260430 19:58:50.754724 4029 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8c60fde179d14eca9132f400b0c28203 has no consensus state
I20260430 19:58:50.766407 4470 leader_election.cc:304] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:50.772684 4911 raft_consensus.cc:2804] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:50.773277 4911 raft_consensus.cc:697] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:50.774343 4911 consensus_queue.cc:237] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.764911 4776 raft_consensus.cc:359] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.782020 4776 raft_consensus.cc:385] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.782380 4776 raft_consensus.cc:740] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
W20260430 19:58:50.782791 4015 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.62:33781, user_credentials={real_user=slave}} blocked reactor thread for 43616.9us
I20260430 19:58:50.783190 4776 consensus_queue.cc:260] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.750516 4818 raft_consensus.cc:359] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.790141 4818 raft_consensus.cc:385] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.790519 4818 raft_consensus.cc:740] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.791213 4818 consensus_queue.cc:260] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.797492 4729 tablet_bootstrap.cc:654] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.806717 4818 ts_tablet_manager.cc:1434] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.065s user 0.007s sys 0.000s
I20260430 19:58:50.811514 4742 ts_tablet_manager.cc:1929] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:50.812005 4742 log.cc:1199] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/13e8206ac4484abfa769276c83fd3333
I20260430 19:58:50.812638 4742 ts_tablet_manager.cc:1950] T 13e8206ac4484abfa769276c83fd3333 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:50.816507 4020 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet d1f7489f1d9c46b8b33745c3e03a2138
I20260430 19:58:50.819993 4818 tablet_bootstrap.cc:492] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:50.823611 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet d1f7489f1d9c46b8b33745c3e03a2138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:50.825124 4818 tablet_bootstrap.cc:654] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:50.826254 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet d1f7489f1d9c46b8b33745c3e03a2138 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1f7489f1d9c46b8b33745c3e03a2138 already in progress: creating tablet
I20260430 19:58:50.827627 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet d1f7489f1d9c46b8b33745c3e03a2138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:50.827678 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet d1f7489f1d9c46b8b33745c3e03a2138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:50.828989 4776 ts_tablet_manager.cc:1434] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.157s user 0.007s sys 0.000s
I20260430 19:58:50.841418 4818 tablet_bootstrap.cc:492] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:50.842907 4818 ts_tablet_manager.cc:1403] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.002s
I20260430 19:58:50.845780 4818 raft_consensus.cc:359] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.847219 4911 raft_consensus.cc:493] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.849153 4818 raft_consensus.cc:385] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.849438 4818 raft_consensus.cc:740] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.850944 4818 consensus_queue.cc:260] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.851353 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_fe61a348f64e4406ab80425071253c23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:50.852989 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe61a348f64e4406ab80425071253c23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:50.854926 4911 raft_consensus.cc:515] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.858151 4960 raft_consensus.cc:493] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.858665 4960 raft_consensus.cc:515] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.858834 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_f025a41a9b8044ba9e39f5634daa2a55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:50.859500 4654 ts_tablet_manager.cc:1929] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:50.859833 4654 log.cc:1199] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/13e8206ac4484abfa769276c83fd3333
W20260430 19:58:50.860258 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f025a41a9b8044ba9e39f5634daa2a55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:50.860581 4654 ts_tablet_manager.cc:1950] T 13e8206ac4484abfa769276c83fd3333 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:50.861305 4960 leader_election.cc:290] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:50.862257 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e87fda401047cfa7070ae95b1701a9" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:50.863998 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e87fda401047cfa7070ae95b1701a9" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:50.865136 4161 leader_election.cc:343] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:50.865783 4158 leader_election.cc:343] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.866189 4158 leader_election.cc:304] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:50.867245 4960 raft_consensus.cc:2749] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:50.874320 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cef0197c00b4ab7a11e2e5123a4c277" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:50.876607 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cef0197c00b4ab7a11e2e5123a4c277" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:50.878636 4470 leader_election.cc:343] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:50.879338 4470 leader_election.cc:343] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.879712 4470 leader_election.cc:304] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:50.880439 4968 raft_consensus.cc:2749] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:50.856976 4911 leader_election.cc:290] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.887205 4776 tablet_bootstrap.cc:492] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:50.896020 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 13e8206ac4484abfa769276c83fd3333 (table default.Table_d38bd1441c3c4a89acfac535b88279a8 [id=25d3d568199f429b8cc145e854ee730b]) successfully deleted
I20260430 19:58:50.903223 4742 tablet_replica.cc:333] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:50.904008 4742 raft_consensus.cc:2243] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:50.904436 4742 raft_consensus.cc:2272] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:50.906200 4742 ts_tablet_manager.cc:1916] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:50.909507 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 13e8206ac4484abfa769276c83fd3333 (table default.Table_d38bd1441c3c4a89acfac535b88279a8 [id=25d3d568199f429b8cc145e854ee730b]) successfully deleted
I20260430 19:58:50.909868 4020 master_service.cc:954] ReplaceTablet: received request to replace tablet ad2ce1d43f214c8d9c1a1001baf53440 from {username='slave'} at 127.0.0.1:55130
W20260430 19:58:50.910267 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 70d410f3b3b04e649b79dda04c0891ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 70d410f3b3b04e649b79dda04c0891ee already in progress: creating tablet
W20260430 19:58:50.910902 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet ba4cf90db44540f2b438966361af8192 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba4cf90db44540f2b438966361af8192 already in progress: creating tablet
W20260430 19:58:50.911471 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet d1f7489f1d9c46b8b33745c3e03a2138 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1f7489f1d9c46b8b33745c3e03a2138 already in progress: creating tablet
W20260430 19:58:50.912223 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet e36490392a404b6bba29bd6803274b36 because the tablet was not found. No further retry: Not found: Tablet not found: e36490392a404b6bba29bd6803274b36
I20260430 19:58:50.914443 4911 raft_consensus.cc:493] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.915320 4818 ts_tablet_manager.cc:1434] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.072s user 0.005s sys 0.000s
I20260430 19:58:50.914942 4911 raft_consensus.cc:515] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.917034 4911 leader_election.cc:290] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.919624 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "654df09f602b4f96bcf5d13a514df55c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:50.920317 4382 raft_consensus.cc:2468] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:50.922511 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "654df09f602b4f96bcf5d13a514df55c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:50.926101 4470 leader_election.cc:343] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.926831 4470 leader_election.cc:304] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:50.926726 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_e8541719669c46e2bcd98610f2c4181b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:50.928664 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_b85a96ef82324def96137ff90837adf1" } new_table_name: "default.Table_cf25ab133dde4d9aa99dabc520b379aa" modify_external_catalogs: true
I20260430 19:58:50.937645 4776 tablet_bootstrap.cc:654] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.941898 4911 raft_consensus.cc:2804] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:50.942255 4911 raft_consensus.cc:493] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:50.942548 4911 raft_consensus.cc:3060] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:50.945218 4818 tablet_bootstrap.cc:492] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
W20260430 19:58:50.945889 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8541719669c46e2bcd98610f2c4181b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:50.957329 4911 raft_consensus.cc:515] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:50.962813 4960 raft_consensus.cc:493] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:50.963287 4960 raft_consensus.cc:515] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.965185 4960 leader_election.cc:290] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:50.968029 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafc058f392a45de9d30b521ebde0950" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:50.968621 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafc058f392a45de9d30b521ebde0950" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:50.969713 4158 leader_election.cc:343] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:50.970326 4161 leader_election.cc:343] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:50.970731 4161 leader_election.cc:304] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:50.971463 4960 raft_consensus.cc:2749] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:50.972571 4729 tablet_bootstrap.cc:492] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:50.973003 4729 ts_tablet_manager.cc:1403] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.239s user 0.014s sys 0.010s
I20260430 19:58:50.975271 4729 raft_consensus.cc:359] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.977844 4729 raft_consensus.cc:385] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.978150 4729 raft_consensus.cc:740] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.978876 4729 consensus_queue.cc:260] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.983300 4818 tablet_bootstrap.cc:654] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:50.983804 4776 tablet_bootstrap.cc:492] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:50.984234 4776 ts_tablet_manager.cc:1403] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.097s user 0.002s sys 0.007s
I20260430 19:58:50.985996 4029 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_345c24b2a5384ad09a15536943d06a18" } modify_external_catalogs: true
I20260430 19:58:50.986569 4029 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_345c24b2a5384ad09a15536943d06a18" } modify_external_catalogs: true
I20260430 19:58:50.986580 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_d87d24b7c4654f5b98b432530730c721" } new_table_name: "default.Table_f79093a8060a4af3979bd0705d8a6998" modify_external_catalogs: true
I20260430 19:58:50.987344 4911 leader_election.cc:290] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:50.986433 4776 raft_consensus.cc:359] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:50.988308 4776 raft_consensus.cc:385] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:50.988548 4776 raft_consensus.cc:740] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:50.992311 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_0cc9e2f04597497a81fd4bf7d8da9734"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:50.992921 4776 consensus_queue.cc:260] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:50.997565 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cc9e2f04597497a81fd4bf7d8da9734 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.004937 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "654df09f602b4f96bcf5d13a514df55c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:51.011274 4776 ts_tablet_manager.cc:1434] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20260430 19:58:51.014712 4776 tablet_bootstrap.cc:492] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
W20260430 19:58:51.022015 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c81b11280f6d49e681537c88fb1cbf68 consensus state has no leader
I20260430 19:58:51.027566 4742 ts_tablet_manager.cc:1929] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:51.028020 4742 log.cc:1199] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/37d6c54ae89946158cad45e741c1c0f0
I20260430 19:58:51.028736 4742 ts_tablet_manager.cc:1950] T 37d6c54ae89946158cad45e741c1c0f0 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:51.029947 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_733dfaaf74774e8db1bfa300e4e56ced" } modify_external_catalogs: true
I20260430 19:58:51.030467 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_733dfaaf74774e8db1bfa300e4e56ced" } modify_external_catalogs: true
I20260430 19:58:51.031462 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 37d6c54ae89946158cad45e741c1c0f0 (table default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]) successfully deleted
W20260430 19:58:51.006485 4470 leader_election.cc:343] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:51.035014 4776 tablet_bootstrap.cc:654] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:51.035598 4729 ts_tablet_manager.cc:1434] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
I20260430 19:58:51.037984 4742 tablet_replica.cc:333] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:51.038796 4742 raft_consensus.cc:2243] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:51.039268 4742 raft_consensus.cc:2272] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:51.041039 4742 ts_tablet_manager.cc:1916] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:51.045900 4729 tablet_bootstrap.cc:492] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:51.003554 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "654df09f602b4f96bcf5d13a514df55c" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:51.048563 4382 raft_consensus.cc:3060] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.055400 4382 raft_consensus.cc:2468] T 654df09f602b4f96bcf5d13a514df55c P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:51.056571 4470 leader_election.cc:304] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe
I20260430 19:58:51.057356 4911 raft_consensus.cc:2804] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:51.057786 4911 raft_consensus.cc:697] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:51.058521 4911 consensus_queue.cc:237] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:51.066198 4729 tablet_bootstrap.cc:654] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:51.080082 4818 tablet_bootstrap.cc:492] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:51.080230 4776 tablet_bootstrap.cc:492] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:51.080693 4818 ts_tablet_manager.cc:1403] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.158s user 0.014s sys 0.000s
I20260430 19:58:51.080711 4776 ts_tablet_manager.cc:1403] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.066s user 0.011s sys 0.003s
I20260430 19:58:51.083186 4818 raft_consensus.cc:359] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:51.083254 4776 raft_consensus.cc:359] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.083949 4776 raft_consensus.cc:385] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:51.084069 4818 raft_consensus.cc:385] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:51.084208 4776 raft_consensus.cc:740] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:51.084483 4818 raft_consensus.cc:740] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:51.084825 4776 consensus_queue.cc:260] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.085147 4818 consensus_queue.cc:260] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:51.090183 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_4e505a72d8d140029f5450a3d9572d4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:51.091753 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e505a72d8d140029f5450a3d9572d4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.098713 4776 ts_tablet_manager.cc:1434] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.018s user 0.007s sys 0.001s
I20260430 19:58:51.102656 4818 ts_tablet_manager.cc:1434] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260430 19:58:51.104485 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_1543a16cba5a42498fd6f031d53ed6a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:51.105947 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1543a16cba5a42498fd6f031d53ed6a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.113831 4818 tablet_bootstrap.cc:492] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:51.115836 4729 tablet_bootstrap.cc:492] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:51.116241 4729 ts_tablet_manager.cc:1403] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.071s user 0.012s sys 0.000s
I20260430 19:58:51.122561 4729 raft_consensus.cc:359] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.123205 4729 raft_consensus.cc:385] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:51.123440 4729 raft_consensus.cc:740] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:51.128235 4776 tablet_bootstrap.cc:492] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:51.131788 4911 raft_consensus.cc:493] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.132220 4911 raft_consensus.cc:515] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.133977 4911 leader_election.cc:290] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.135536 4742 ts_tablet_manager.cc:1929] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:51.135896 4742 log.cc:1199] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/ad2ce1d43f214c8d9c1a1001baf53440
I20260430 19:58:51.136072 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c119dd9a5d944eabd588f62a76da98e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:51.136452 4742 ts_tablet_manager.cc:1950] T ad2ce1d43f214c8d9c1a1001baf53440 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:51.137116 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c119dd9a5d944eabd588f62a76da98e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:51.137641 4223 raft_consensus.cc:2468] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:51.139072 4470 leader_election.cc:304] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
W20260430 19:58:51.140424 4470 leader_election.cc:343] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:51.141197 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet ad2ce1d43f214c8d9c1a1001baf53440 (table default.Table_cf6f9cf5be37436f9671fa67edc046a1 [id=043f0b9bc9274b9fbdbdd17fb0d6689c]) successfully deleted
I20260430 19:58:51.141831 4911 raft_consensus.cc:2804] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:51.141947 4776 tablet_bootstrap.cc:654] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:51.142149 4911 raft_consensus.cc:493] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:51.142441 4911 raft_consensus.cc:3060] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.144438 4742 tablet_replica.cc:333] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:51.145115 4742 raft_consensus.cc:2243] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:51.145500 4742 raft_consensus.cc:2272] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:51.147073 4742 ts_tablet_manager.cc:1916] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:51.160027 4776 tablet_bootstrap.cc:492] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:51.160490 4776 ts_tablet_manager.cc:1403] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.006s
I20260430 19:58:51.162791 4776 raft_consensus.cc:359] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.163455 4776 raft_consensus.cc:385] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:51.163705 4776 raft_consensus.cc:740] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:51.165639 4776 consensus_queue.cc:260] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.171394 4729 consensus_queue.cc:260] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.172224 4742 ts_tablet_manager.cc:1929] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:51.172005 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_d155c3e7331947ccb6d24c2244ec2537"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:51.172551 4742 log.cc:1199] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/e4e87fda401047cfa7070ae95b1701a9
I20260430 19:58:51.173107 4742 ts_tablet_manager.cc:1950] T e4e87fda401047cfa7070ae95b1701a9 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:51.153023 4911 raft_consensus.cc:515] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:51.173436 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d155c3e7331947ccb6d24c2244ec2537 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.175348 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c119dd9a5d944eabd588f62a76da98e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:51.177263 4729 ts_tablet_manager.cc:1434] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.061s user 0.006s sys 0.001s
I20260430 19:58:51.179150 4911 leader_election.cc:290] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.179802 4818 tablet_bootstrap.cc:654] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:51.182507 4729 tablet_bootstrap.cc:492] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
W20260430 19:58:51.186154 4470 leader_election.cc:343] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:51.186848 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c119dd9a5d944eabd588f62a76da98e" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:51.187350 4223 raft_consensus.cc:3060] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.187755 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 6f39fb64a301447f988c53ccfe896074 (DEFAULT_TABLE table=default.Table_f01cb568b13b4e5ab73b36deda0357f3 [id=39b79f4586ea427b95c6fdfff58dcea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.188822 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f39fb64a301447f988c53ccfe896074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.193737 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 6f39fb64a301447f988c53ccfe896074 (DEFAULT_TABLE table=default.Table_f01cb568b13b4e5ab73b36deda0357f3 [id=39b79f4586ea427b95c6fdfff58dcea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.182425 4021 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:51.194519 4021 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:51.194743 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f39fb64a301447f988c53ccfe896074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.195040 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 6f39fb64a301447f988c53ccfe896074 (DEFAULT_TABLE table=default.Table_f01cb568b13b4e5ab73b36deda0357f3 [id=39b79f4586ea427b95c6fdfff58dcea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.196103 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f39fb64a301447f988c53ccfe896074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.195652 4021 catalog_manager.cc:5671] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } }
I20260430 19:58:51.196964 4021 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
W20260430 19:58:51.200719 4470 consensus_peers.cc:597] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13e8206ac4484abfa769276c83fd3333. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:51.204121 4223 raft_consensus.cc:2468] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:51.206447 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 4758b3d846434b2b8466033c083b7339 (DEFAULT_TABLE table=default.Table_c8f20997cfad46fea29e3a23559df569 [id=8e98bcce763f4a528fa2299c59d34221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.207695 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4758b3d846434b2b8466033c083b7339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.211319 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 4758b3d846434b2b8466033c083b7339 (DEFAULT_TABLE table=default.Table_c8f20997cfad46fea29e3a23559df569 [id=8e98bcce763f4a528fa2299c59d34221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.212034 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 66e94750de27416e9adcf60fccf2ace9 (DEFAULT_TABLE table=default.Table_5b5580edb7844ccb8271ad24e96a1eb0 [id=245a6d903af145969e0ff7d40eb5f1e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.212316 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4758b3d846434b2b8466033c083b7339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.213035 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66e94750de27416e9adcf60fccf2ace9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.227427 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 66e94750de27416e9adcf60fccf2ace9 (DEFAULT_TABLE table=default.Table_5b5580edb7844ccb8271ad24e96a1eb0 [id=245a6d903af145969e0ff7d40eb5f1e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.227607 4020 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_a7e1ecd9b8b44627a65e244546a87c58" } new_table_name: "default.Table_4f04fa211ec3447488b05b4f5b157487" modify_external_catalogs: true
I20260430 19:58:51.228549 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66e94750de27416e9adcf60fccf2ace9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.229493 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 081a060c44924bcda09ffc8fe88012b3
I20260430 19:58:51.231501 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 081a060c44924bcda09ffc8fe88012b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:51.231679 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet 081a060c44924bcda09ffc8fe88012b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:51.233007 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 081a060c44924bcda09ffc8fe88012b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 081a060c44924bcda09ffc8fe88012b3 already in progress: creating tablet
W20260430 19:58:51.233829 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 081a060c44924bcda09ffc8fe88012b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 081a060c44924bcda09ffc8fe88012b3 already in progress: creating tablet
I20260430 19:58:51.207387 4196 tablet_service.cc:1511] Processing CreateTablet for tablet 4758b3d846434b2b8466033c083b7339 (DEFAULT_TABLE table=default.Table_c8f20997cfad46fea29e3a23559df569 [id=8e98bcce763f4a528fa2299c59d34221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.258494 4023 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:51.259176 4023 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet c78d50690da7428fadb848dadefec47c (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2]). Expected version 4 got 0
I20260430 19:58:51.259372 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4758b3d846434b2b8466033c083b7339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.221292 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 66e94750de27416e9adcf60fccf2ace9 (DEFAULT_TABLE table=default.Table_5b5580edb7844ccb8271ad24e96a1eb0 [id=245a6d903af145969e0ff7d40eb5f1e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.263296 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66e94750de27416e9adcf60fccf2ace9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.231231 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet 081a060c44924bcda09ffc8fe88012b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:51.271354 4470 leader_election.cc:304] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:51.272380 4911 raft_consensus.cc:2804] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:51.275935 4911 raft_consensus.cc:697] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:51.276861 4911 consensus_queue.cc:237] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.292888 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e8206ac4484abfa769276c83fd3333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:51.293275 4776 ts_tablet_manager.cc:1434] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.132s user 0.006s sys 0.001s
I20260430 19:58:51.293617 4915 tablet_replica.cc:333] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:51.294368 4915 raft_consensus.cc:2243] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:51.295178 4915 pending_rounds.cc:70] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Trying to abort 1 pending ops.
I20260430 19:58:51.295411 4915 pending_rounds.cc:77] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964117604552704 op_type: NO_OP noop_request { }
I20260430 19:58:51.295786 4915 raft_consensus.cc:2889] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:51.296109 4915 raft_consensus.cc:2272] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:51.297922 4915 ts_tablet_manager.cc:1916] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:51.303171 4776 tablet_bootstrap.cc:492] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
W20260430 19:58:51.305363 4021 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.3:35079 (request call id 35) took 894 ms (client timeout 1000 ms). Trace:
I20260430 19:58:51.305749 4729 tablet_bootstrap.cc:700] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent opening tablet: real 0.119s user 0.000s sys 0.000s
I20260430 19:58:51.306314 4729 tablet_bootstrap.cc:654] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:51.305768 4021 rpcz_store.cc:269] 0430 19:58:50.410978 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:50.457415 (+ 46437us) service_pool.cc:225] Handling call
0430 19:58:51.305304 (+847889us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:51.207846 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:51.208219 (+ 373us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:51.208243 (+ 24us) write_op.cc:435] Acquired schema lock
0430 19:58:51.208257 (+ 14us) tablet.cc:662] Decoding operations
0430 19:58:51.208533 (+ 276us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:51.208565 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:51.208586 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:51.208715 (+ 129us) tablet.cc:701] Row locks acquired
0430 19:58:51.208726 (+ 11us) write_op.cc:260] PREPARE: finished
0430 19:58:51.208827 (+ 101us) write_op.cc:270] Start()
0430 19:58:51.208910 (+ 83us) write_op.cc:276] Timestamp: P: 1777579131208780 usec, L: 0
0430 19:58:51.208925 (+ 15us) op_driver.cc:348] REPLICATION: starting
0430 19:58:51.209405 (+ 480us) log.cc:844] Serialized 527 byte log entry
0430 19:58:51.237344 (+ 27939us) op_driver.cc:464] REPLICATION: finished
0430 19:58:51.241023 (+ 3679us) write_op.cc:301] APPLY: starting
0430 19:58:51.241069 (+ 46us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:51.241152 (+ 83us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:51.241232 (+ 80us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:51.241400 (+ 168us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:51.241514 (+ 114us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:51.241552 (+ 38us) write_op.cc:312] APPLY: finished
0430 19:58:51.241692 (+ 140us) log.cc:844] Serialized 23 byte log entry
0430 19:58:51.241801 (+ 109us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:51.241849 (+ 48us) write_op.cc:454] Released schema lock
0430 19:58:51.241873 (+ 24us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":3551,"apply.run_cpu_time_us":1014,"apply.run_wall_time_us":1015,"num_ops":1,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":1806,"prepare.run_wall_time_us":1935,"raft.queue_time_us":72,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":28353,"threads_started":1}]]}
I20260430 19:58:51.318212 4911 raft_consensus.cc:493] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.319231 4911 raft_consensus.cc:515] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.322434 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb14746d8564bf8833e933a01771492" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:51.323019 4382 raft_consensus.cc:2468] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:51.323870 4470 leader_election.cc:304] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:51.324596 4912 raft_consensus.cc:2804] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:51.324903 4912 raft_consensus.cc:493] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:51.325187 4912 raft_consensus.cc:3060] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.314488 5010 raft_consensus.cc:493] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.327538 5010 raft_consensus.cc:515] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.329558 5010 leader_election.cc:290] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.332352 4912 raft_consensus.cc:515] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.335232 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb14746d8564bf8833e933a01771492" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:51.335976 4223 raft_consensus.cc:2468] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:51.336766 4818 tablet_bootstrap.cc:492] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:51.337226 4818 ts_tablet_manager.cc:1403] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.224s user 0.007s sys 0.003s
I20260430 19:58:51.338940 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb14746d8564bf8833e933a01771492" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:51.339442 4382 raft_consensus.cc:3060] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.335851 4911 leader_election.cc:290] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.341831 4818 raft_consensus.cc:359] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.342417 4818 raft_consensus.cc:385] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:51.342689 4818 raft_consensus.cc:740] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:51.342908 4912 leader_election.cc:290] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.342787 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb14746d8564bf8833e933a01771492" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:51.343400 4223 raft_consensus.cc:3060] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.343222 4818 consensus_queue.cc:260] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.353065 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:51.353902 4382 raft_consensus.cc:2468] T edb14746d8564bf8833e933a01771492 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:51.360765 4470 leader_election.cc:304] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:51.361616 4912 raft_consensus.cc:2804] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:51.361910 4203 tablet_service.cc:1511] Processing CreateTablet for tablet bdc476c6909d4c998f8b102029eb62a4 (DEFAULT_TABLE table=default.Table_81ac9df8ed4947d8bae7f45d6dfef943 [id=af1133bdbccd4e8bb61b4eac1197b2bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.362891 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc476c6909d4c998f8b102029eb62a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.362082 4912 raft_consensus.cc:697] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:51.365793 4911 raft_consensus.cc:493] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.366173 4911 raft_consensus.cc:515] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.366477 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 5563cfa73f194db69a6978d348f1c073 (DEFAULT_TABLE table=default.Table_345c24b2a5384ad09a15536943d06a18 [id=e6d3bdf7535049998b8f7593e582d02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.367338 4776 tablet_bootstrap.cc:654] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:51.367503 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5563cfa73f194db69a6978d348f1c073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.367996 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0832f9191b7f4c3683f37cce8966b266" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:51.368687 4225 raft_consensus.cc:2393] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8345ec7f57c4399a8a4afe4dd43439e in current term 1: Already voted for candidate 8a11a9a7f244435b8e84554e3fabbd0f in this term.
I20260430 19:58:51.370003 4223 raft_consensus.cc:2468] T edb14746d8564bf8833e933a01771492 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:51.371618 4818 ts_tablet_manager.cc:1434] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.034s user 0.007s sys 0.000s
I20260430 19:58:51.372509 4818 tablet_bootstrap.cc:492] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:51.372910 4360 tablet_service.cc:1558] Processing DeleteTablet for tablet ad2ce1d43f214c8d9c1a1001baf53440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:51.373368 4320 leader_election.cc:304] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:51.373775 4357 tablet_service.cc:1511] Processing CreateTablet for tablet bdc476c6909d4c998f8b102029eb62a4 (DEFAULT_TABLE table=default.Table_81ac9df8ed4947d8bae7f45d6dfef943 [id=af1133bdbccd4e8bb61b4eac1197b2bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.375367 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 5563cfa73f194db69a6978d348f1c073 (DEFAULT_TABLE table=default.Table_345c24b2a5384ad09a15536943d06a18 [id=e6d3bdf7535049998b8f7593e582d02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.376727 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 5563cfa73f194db69a6978d348f1c073 (DEFAULT_TABLE table=default.Table_345c24b2a5384ad09a15536943d06a18 [id=e6d3bdf7535049998b8f7593e582d02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.377789 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5563cfa73f194db69a6978d348f1c073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.378942 4912 consensus_queue.cc:237] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:51.381296 4470 leader_election.cc:343] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: 37d6c54ae89946158cad45e741c1c0f0
I20260430 19:58:51.381662 5014 raft_consensus.cc:3060] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.383122 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc476c6909d4c998f8b102029eb62a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.371312 4911 leader_election.cc:290] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.320987 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_9cfed595445342da8d6b800243a1a693"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:51.390162 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5563cfa73f194db69a6978d348f1c073. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:51.390568 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cfed595445342da8d6b800243a1a693 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.321228 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_dfb4852e9eaa4eb482610e15c43c2e87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:51.395136 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfb4852e9eaa4eb482610e15c43c2e87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.323800 4028 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 9d1b860c7073446597ce696f822ce1c2 (table default.Table_cf25ab133dde4d9aa99dabc520b379aa [id=3a1eab4e333c4999b8b5155702ad0e3b]). Expected version 1 got 0
W20260430 19:58:51.330693 4020 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 51e35e0981e649f684caa96499040c15 consensus state has no leader
I20260430 19:58:51.331954 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 2c119dd9a5d944eabd588f62a76da98e
I20260430 19:58:51.402232 4025 catalog_manager.cc:6741] ReplaceTablet: tablet 2c119dd9a5d944eabd588f62a76da98e deleted and replaced by tablet 239a2c6a534f4f478e436d14f94cd762
I20260430 19:58:51.344075 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_6fbda673636e42419e82c47142a45243"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:51.403139 4513 tablet_service.cc:1511] Processing CreateTablet for tablet bdc476c6909d4c998f8b102029eb62a4 (DEFAULT_TABLE table=default.Table_81ac9df8ed4947d8bae7f45d6dfef943 [id=af1133bdbccd4e8bb61b4eac1197b2bb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:51.403848 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6fbda673636e42419e82c47142a45243 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.404070 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc476c6909d4c998f8b102029eb62a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.404763 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37d6c54ae89946158cad45e741c1c0f0" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:51.405814 4470 leader_election.cc:343] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:51.406158 4470 leader_election.cc:304] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:51.348448 4029 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 5563cfa73f194db69a6978d348f1c073
I20260430 19:58:51.407151 4911 raft_consensus.cc:2749] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:51.371297 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet e4e87fda401047cfa7070ae95b1701a9 (table default.Table_4b6fc137cf6945079fa8310db94a2b1a [id=2cb52e96af9f4eea998718eef0ac58ad]) successfully deleted
W20260430 19:58:51.385571 4015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260430 19:58:51.413738 4205 tablet_service.cc:1511] Processing CreateTablet for tablet adf6f4bb3e16430a8a4b10dea4872c98 (DEFAULT_TABLE table=default.Table_2a59def4cb444084bf2b70663d1430b4 [id=a48728cdc32e4dfa8a5711bb7837cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.414698 4204 tablet_service.cc:1558] Processing DeleteTablet for tablet 5563cfa73f194db69a6978d348f1c073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:51.415513 4968 consensus_queue.cc:1048] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:51.417495 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c119dd9a5d944eabd588f62a76da98e with delete_type TABLET_DATA_DELETED (replaced by tablet 239a2c6a534f4f478e436d14f94cd762) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:51.418457 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet 5563cfa73f194db69a6978d348f1c073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:51.418926 4470 consensus_peers.cc:597] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:51.419936 4503 tablet_service.cc:1511] Processing CreateTablet for tablet adf6f4bb3e16430a8a4b10dea4872c98 (DEFAULT_TABLE table=default.Table_2a59def4cb444084bf2b70663d1430b4 [id=a48728cdc32e4dfa8a5711bb7837cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.420547 4915 ts_tablet_manager.cc:1929] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:51.420871 4915 log.cc:1199] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/13e8206ac4484abfa769276c83fd3333
I20260430 19:58:51.421487 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf6f4bb3e16430a8a4b10dea4872c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.421603 4915 ts_tablet_manager.cc:1950] T 13e8206ac4484abfa769276c83fd3333 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:51.424789 5014 raft_consensus.cc:2749] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:51.418912 4514 tablet_service.cc:1511] Processing CreateTablet for tablet db66fb6e6e654d7098ee69899bfb6e34 (DEFAULT_TABLE table=default.Table_f0911e2389af4d0c90e177647db6ee84 [id=187a06e1eb3242f485abed439a632efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.427835 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db66fb6e6e654d7098ee69899bfb6e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.437181 4025 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_2afe6c2b4071491f868a3a17cc4d6e48" } modify_external_catalogs: true
I20260430 19:58:51.437722 4025 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_2afe6c2b4071491f868a3a17cc4d6e48" } modify_external_catalogs: true
I20260430 19:58:51.414693 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf6f4bb3e16430a8a4b10dea4872c98. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:51.446334 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet ad2ce1d43f214c8d9c1a1001baf53440 because the tablet was not found. No further retry: Not found: Tablet not found: ad2ce1d43f214c8d9c1a1001baf53440
I20260430 19:58:51.446439 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_659324304ce3463fa7e7306b42d7ffb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:51.446970 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 70d410f3b3b04e649b79dda04c0891ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 70d410f3b3b04e649b79dda04c0891ee already in progress: creating tablet
W20260430 19:58:51.447503 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet ba4cf90db44540f2b438966361af8192 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba4cf90db44540f2b438966361af8192 already in progress: creating tablet
W20260430 19:58:51.447885 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_659324304ce3463fa7e7306b42d7ffb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:51.448069 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet d1f7489f1d9c46b8b33745c3e03a2138 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d1f7489f1d9c46b8b33745c3e03a2138 already in progress: creating tablet
W20260430 19:58:51.448581 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 081a060c44924bcda09ffc8fe88012b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 081a060c44924bcda09ffc8fe88012b3 already in progress: creating tablet
W20260430 19:58:51.449211 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 5563cfa73f194db69a6978d348f1c073 because the tablet was not found. No further retry: Not found: Tablet not found: 5563cfa73f194db69a6978d348f1c073
I20260430 19:58:51.450227 4196 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c119dd9a5d944eabd588f62a76da98e with delete_type TABLET_DATA_DELETED (replaced by tablet 239a2c6a534f4f478e436d14f94cd762) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:51.450910 4742 tablet_replica.cc:333] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:51.451570 4742 raft_consensus.cc:2243] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:51.451972 4742 raft_consensus.cc:2272] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:51.453537 4742 ts_tablet_manager.cc:1916] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:51.460450 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet 443308f2a26f4622aa4551ff5f73fc39 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:51.463069 4028 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet c81b11280f6d49e681537c88fb1cbf68 (table default.Table_f79093a8060a4af3979bd0705d8a6998 [id=7b43b6a692214e0b89c73b9374682197]). Expected version 3 got 0
I20260430 19:58:51.463670 4028 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:51.465844 4023 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 404f20adf9d741f3a036b06cbdbaedb8 (table default.Table_2afe6c2b4071491f868a3a17cc4d6e48 [id=06859fba4a5d4b8f978e6f56edc5fb85]). Expected version 2 got 0
I20260430 19:58:51.466503 4023 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:51.467123 4023 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 0832f9191b7f4c3683f37cce8966b266 (table default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]). Expected version 1 got 0
I20260430 19:58:51.469295 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_da91e4435ef64c98bc07306bcb50023f" } new_table_name: "default.Table_47de4e0e38f64d868f55414d6a9a4f6f" modify_external_catalogs: true
I20260430 19:58:51.478879 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 13e8206ac4484abfa769276c83fd3333 (table default.Table_d38bd1441c3c4a89acfac535b88279a8 [id=25d3d568199f429b8cc145e854ee730b]) successfully deleted
I20260430 19:58:51.494247 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 8c60fde179d14eca9132f400b0c28203 (DEFAULT_TABLE table=default.Table_04b53e66a05c4462860e4411b8eb2312 [id=754aa5df4c4548afbafb001eecf30387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.495337 4193 tablet_service.cc:1511] Processing CreateTablet for tablet db66fb6e6e654d7098ee69899bfb6e34 (DEFAULT_TABLE table=default.Table_f0911e2389af4d0c90e177647db6ee84 [id=187a06e1eb3242f485abed439a632efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.496596 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db66fb6e6e654d7098ee69899bfb6e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.496431 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 0cef4984c7de491bbe64f97c8d7ff044 (DEFAULT_TABLE table=default.Table_a989a7af579548079a4722aee571d7b4 [id=73a72d00843043198cfa5e155f23e7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.497881 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cef4984c7de491bbe64f97c8d7ff044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.498795 4502 tablet_service.cc:1511] Processing CreateTablet for tablet ac46dc3dad9c48a9b4282982e7c1bc8d (DEFAULT_TABLE table=default.Table_acaae4cbbb6443049cb4c80749caf848 [id=4fb19f06eb5f47f787db6b6f73ce91ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.500038 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac46dc3dad9c48a9b4282982e7c1bc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.505345 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_da91e4435ef64c98bc07306bcb50023f [id=91e641e6e87f45ffb0a050142b221074]). Expected version 1 got 0
I20260430 19:58:51.507988 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 8c60fde179d14eca9132f400b0c28203 (DEFAULT_TABLE table=default.Table_04b53e66a05c4462860e4411b8eb2312 [id=754aa5df4c4548afbafb001eecf30387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.509131 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c60fde179d14eca9132f400b0c28203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.509541 4196 tablet_service.cc:1511] Processing CreateTablet for tablet 0cef4984c7de491bbe64f97c8d7ff044 (DEFAULT_TABLE table=default.Table_a989a7af579548079a4722aee571d7b4 [id=73a72d00843043198cfa5e155f23e7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.510691 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cef4984c7de491bbe64f97c8d7ff044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.519261 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c60fde179d14eca9132f400b0c28203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.530925 4915 tablet_replica.cc:333] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:51.531919 4915 raft_consensus.cc:2243] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:51.532593 4915 pending_rounds.cc:70] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Trying to abort 1 pending ops.
I20260430 19:58:51.532546 4203 tablet_service.cc:1511] Processing CreateTablet for tablet ac46dc3dad9c48a9b4282982e7c1bc8d (DEFAULT_TABLE table=default.Table_acaae4cbbb6443049cb4c80749caf848 [id=4fb19f06eb5f47f787db6b6f73ce91ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.532799 4915 pending_rounds.cc:77] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964121720410112 op_type: NO_OP noop_request { }
I20260430 19:58:51.533128 4915 raft_consensus.cc:2889] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:51.533412 4915 raft_consensus.cc:2272] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:51.533996 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac46dc3dad9c48a9b4282982e7c1bc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.535708 4915 ts_tablet_manager.cc:1916] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:51.544070 4729 tablet_bootstrap.cc:492] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:51.544575 4729 ts_tablet_manager.cc:1403] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.362s user 0.012s sys 0.002s
I20260430 19:58:51.554955 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 50d8b9a982fe4810858d1ec2b28d7894 (DEFAULT_TABLE table=default.Table_9312a99c3d0040e3a6c62e6400b06041 [id=d201cb9d223442ba9fca28fa5112fac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.555972 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d8b9a982fe4810858d1ec2b28d7894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.555893 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
I20260430 19:58:51.556842 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:51.568449 4352 tablet_service.cc:1511] Processing CreateTablet for tablet adf6f4bb3e16430a8a4b10dea4872c98 (DEFAULT_TABLE table=default.Table_2a59def4cb444084bf2b70663d1430b4 [id=a48728cdc32e4dfa8a5711bb7837cae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.569591 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf6f4bb3e16430a8a4b10dea4872c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.570315 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 8c60fde179d14eca9132f400b0c28203 (DEFAULT_TABLE table=default.Table_04b53e66a05c4462860e4411b8eb2312 [id=754aa5df4c4548afbafb001eecf30387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.570794 4361 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c119dd9a5d944eabd588f62a76da98e with delete_type TABLET_DATA_DELETED (replaced by tablet 239a2c6a534f4f478e436d14f94cd762) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:51.571329 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c60fde179d14eca9132f400b0c28203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.571774 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 0cef4984c7de491bbe64f97c8d7ff044 (DEFAULT_TABLE table=default.Table_a989a7af579548079a4722aee571d7b4 [id=73a72d00843043198cfa5e155f23e7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.572649 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cef4984c7de491bbe64f97c8d7ff044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.576681 4360 tablet_service.cc:1511] Processing CreateTablet for tablet db66fb6e6e654d7098ee69899bfb6e34 (DEFAULT_TABLE table=default.Table_f0911e2389af4d0c90e177647db6ee84 [id=187a06e1eb3242f485abed439a632efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.579233 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db66fb6e6e654d7098ee69899bfb6e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.583263 4351 tablet_service.cc:1558] Processing DeleteTablet for tablet 5563cfa73f194db69a6978d348f1c073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:51.583775 4354 tablet_service.cc:1511] Processing CreateTablet for tablet ac46dc3dad9c48a9b4282982e7c1bc8d (DEFAULT_TABLE table=default.Table_acaae4cbbb6443049cb4c80749caf848 [id=4fb19f06eb5f47f787db6b6f73ce91ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.593214 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac46dc3dad9c48a9b4282982e7c1bc8d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:51.593837 4013 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet d1d52994ff694886b7d4ff4224367082 (table default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet d1d52994ff694886b7d4ff4224367082 consensus state has no leader
I20260430 19:58:51.597425 4911 consensus_queue.cc:1048] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:51.608192 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 34a6d662493e428aa475979abbffc9d7 (DEFAULT_TABLE table=default.Table_028cf8f724c64a51adefa0cc9c15a50e [id=cd706c6a34ac49a4b82e411aca27698a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.618185 4026 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_a7ec0d6efc404f0d80ccdee5fc87c213" } new_table_name: "default.Table_15c9bde5c7f54d888d9c298975bd5e06" modify_external_catalogs: true
I20260430 19:58:51.616129 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a6d662493e428aa475979abbffc9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.622536 4024 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_f79093a8060a4af3979bd0705d8a6998" } modify_external_catalogs: true
I20260430 19:58:51.623179 4024 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_f79093a8060a4af3979bd0705d8a6998" } modify_external_catalogs: true
I20260430 19:58:51.625876 4911 consensus_queue.cc:1048] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:51.633368 4915 ts_tablet_manager.cc:1929] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:51.636627 4915 log.cc:1199] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/2c119dd9a5d944eabd588f62a76da98e
I20260430 19:58:51.636987 4776 tablet_bootstrap.cc:492] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:51.637326 4915 ts_tablet_manager.cc:1950] T 2c119dd9a5d944eabd588f62a76da98e P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:51.637382 4776 ts_tablet_manager.cc:1403] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.334s user 0.008s sys 0.003s
I20260430 19:58:51.640007 4776 raft_consensus.cc:359] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.640837 4776 raft_consensus.cc:385] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:51.641184 4776 raft_consensus.cc:740] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:51.673445 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 2c119dd9a5d944eabd588f62a76da98e (table default.Table_445a32d80e864772b0a615ef426ccadb [id=51ad748be7c14d32ac99d6f8f63f569e]) successfully deleted
W20260430 19:58:51.674122 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 5563cfa73f194db69a6978d348f1c073 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5563cfa73f194db69a6978d348f1c073 already in progress: creating tablet
I20260430 19:58:51.676786 4818 tablet_bootstrap.cc:654] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:51.557508 4029 catalog_manager.cc:5671] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } }
I20260430 19:58:51.679654 4029 catalog_manager.cc:5671] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:51.547680 4729 raft_consensus.cc:359] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:51.680754 4729 raft_consensus.cc:385] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:51.681025 4729 raft_consensus.cc:740] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:51.680953 4029 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 0832f9191b7f4c3683f37cce8966b266 (table default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]). Expected version 1 got 0
I20260430 19:58:51.682787 4505 tablet_service.cc:1511] Processing CreateTablet for tablet bdf98515d8284e3b9b84480807e8108a (DEFAULT_TABLE table=default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.683849 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf98515d8284e3b9b84480807e8108a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:51.684181 4016 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.61:35559, user_credentials={real_user=slave}} blocked reactor thread for 152302us
I20260430 19:58:51.685966 4354 tablet_service.cc:1511] Processing CreateTablet for tablet 50d8b9a982fe4810858d1ec2b28d7894 (DEFAULT_TABLE table=default.Table_9312a99c3d0040e3a6c62e6400b06041 [id=d201cb9d223442ba9fca28fa5112fac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.634543 5034 tablet.cc:1724] T 9d1b860c7073446597ce696f822ce1c2 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:51.689836 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d8b9a982fe4810858d1ec2b28d7894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.681708 4729 consensus_queue.cc:260] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:51.697244 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 2c119dd9a5d944eabd588f62a76da98e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c119dd9a5d944eabd588f62a76da98e already in progress: creating tablet
W20260430 19:58:51.697855 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 5563cfa73f194db69a6978d348f1c073 because the tablet was not found. No further retry: Not found: Tablet not found: 5563cfa73f194db69a6978d348f1c073
I20260430 19:58:51.704597 4729 ts_tablet_manager.cc:1434] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.160s user 0.005s sys 0.001s
I20260430 19:58:51.707393 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 34a6d662493e428aa475979abbffc9d7 (DEFAULT_TABLE table=default.Table_028cf8f724c64a51adefa0cc9c15a50e [id=cd706c6a34ac49a4b82e411aca27698a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.711227 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a6d662493e428aa475979abbffc9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.713994 4729 tablet_bootstrap.cc:492] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:51.720113 4514 tablet_service.cc:1511] Processing CreateTablet for tablet e9927db780674f31be8f043fe0843736 (DEFAULT_TABLE table=default.Table_8ebf726da3884df1b2b6805cb6b147a5 [id=54576f526a0c449e90b1e8c86326b803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.641960 4776 consensus_queue.cc:260] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.729915 5032 raft_consensus.cc:493] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.730526 5032 raft_consensus.cc:515] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.731029 4742 ts_tablet_manager.cc:1929] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:51.731377 4742 log.cc:1199] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/2c119dd9a5d944eabd588f62a76da98e
I20260430 19:58:51.731977 4742 ts_tablet_manager.cc:1950] T 2c119dd9a5d944eabd588f62a76da98e P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:51.732342 5032 leader_election.cc:290] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:51.735260 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 50d8b9a982fe4810858d1ec2b28d7894 (DEFAULT_TABLE table=default.Table_9312a99c3d0040e3a6c62e6400b06041 [id=d201cb9d223442ba9fca28fa5112fac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.736316 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d8b9a982fe4810858d1ec2b28d7894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.737244 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9927db780674f31be8f043fe0843736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.744338 4363 tablet_service.cc:1511] Processing CreateTablet for tablet bdf98515d8284e3b9b84480807e8108a (DEFAULT_TABLE table=default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.745301 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf98515d8284e3b9b84480807e8108a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.745296 5014 raft_consensus.cc:493] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.750990 5014 raft_consensus.cc:515] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.752095 4352 tablet_service.cc:1511] Processing CreateTablet for tablet e9927db780674f31be8f043fe0843736 (DEFAULT_TABLE table=default.Table_8ebf726da3884df1b2b6805cb6b147a5 [id=54576f526a0c449e90b1e8c86326b803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.752982 5014 leader_election.cc:290] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.755455 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9927db780674f31be8f043fe0843736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.744508 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6220177b3e3e4d18bd8e20e6d9d96be7" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:51.742172 4729 tablet_bootstrap.cc:654] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:51.754032 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cef0197c00b4ab7a11e2e5123a4c277" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:51.767022 4530 raft_consensus.cc:2468] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:51.768396 4317 leader_election.cc:304] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:51.769248 5014 raft_consensus.cc:2804] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:51.769634 5014 raft_consensus.cc:493] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:51.769922 5014 raft_consensus.cc:3060] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.776932 5014 raft_consensus.cc:515] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.780320 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cef0197c00b4ab7a11e2e5123a4c277" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:51.780871 4222 raft_consensus.cc:3060] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.783972 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cef0197c00b4ab7a11e2e5123a4c277" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:51.786484 4028 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_58ec6ea9c8dc4e2c9d35ab8648add502 [id=94190b075a2a4a828681f5bb6594f8a2] alter complete (version 4)
I20260430 19:58:51.787020 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 404f20adf9d741f3a036b06cbdbaedb8
I20260430 19:58:51.791450 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 404f20adf9d741f3a036b06cbdbaedb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:51.792075 4915 tablet_replica.cc:333] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:51.792809 4915 raft_consensus.cc:2243] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:51.793252 4915 raft_consensus.cc:2272] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:51.793843 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cef0197c00b4ab7a11e2e5123a4c277" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:51.794359 4530 raft_consensus.cc:3060] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.795361 4915 ts_tablet_manager.cc:1916] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:51.794430 5014 leader_election.cc:290] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.802505 4505 tablet_service.cc:1511] Processing CreateTablet for tablet e12d0f56f8db41aa9a1ba3f23e6428b5 (DEFAULT_TABLE table=default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.803007 4911 raft_consensus.cc:493] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.803521 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12d0f56f8db41aa9a1ba3f23e6428b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.803411 4911 raft_consensus.cc:515] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.805184 4911 leader_election.cc:290] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.806824 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09f272340cd942e9b2af9e38a2bdec82" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:51.808516 4470 leader_election.cc:343] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:51.809223 4016 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.61:35559, user_credentials={real_user=slave}} blocked reactor thread for 91795.6us
I20260430 19:58:51.813005 4915 ts_tablet_manager.cc:1929] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:51.813352 4915 log.cc:1199] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/404f20adf9d741f3a036b06cbdbaedb8
I20260430 19:58:51.814963 4915 ts_tablet_manager.cc:1950] T 404f20adf9d741f3a036b06cbdbaedb8 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:51.815260 4776 ts_tablet_manager.cc:1434] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.178s user 0.003s sys 0.010s
I20260430 19:58:51.816133 4776 tablet_bootstrap.cc:492] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:51.817813 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 404f20adf9d741f3a036b06cbdbaedb8 (table default.Table_2afe6c2b4071491f868a3a17cc4d6e48 [id=06859fba4a5d4b8f978e6f56edc5fb85]) successfully deleted
I20260430 19:58:51.815441 4222 raft_consensus.cc:2468] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:51.819892 4320 leader_election.cc:304] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:51.820789 5014 raft_consensus.cc:2804] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:51.821271 5014 raft_consensus.cc:697] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
W20260430 19:58:51.817081 4161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 51989us
W20260430 19:58:51.822320 4161 leader_election.cc:343] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:51.822160 5014 consensus_queue.cc:237] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.823463 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09f272340cd942e9b2af9e38a2bdec82" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:51.832316 4222 raft_consensus.cc:2468] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:51.835302 4198 tablet_service.cc:1558] Processing DeleteTablet for tablet 404f20adf9d741f3a036b06cbdbaedb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:51.818635 4354 tablet_service.cc:1558] Processing DeleteTablet for tablet 404f20adf9d741f3a036b06cbdbaedb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:51.839102 4654 tablet_replica.cc:333] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:51.839787 4654 raft_consensus.cc:2243] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:51.840186 4654 raft_consensus.cc:2272] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:51.831720 4158 leader_election.cc:343] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 6220177b3e3e4d18bd8e20e6d9d96be7
I20260430 19:58:51.830458 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 34a6d662493e428aa475979abbffc9d7 (DEFAULT_TABLE table=default.Table_028cf8f724c64a51adefa0cc9c15a50e [id=cd706c6a34ac49a4b82e411aca27698a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.840910 4470 leader_election.cc:304] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:51.841189 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a6d662493e428aa475979abbffc9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.831521 4190 tablet_service.cc:1511] Processing CreateTablet for tablet e12d0f56f8db41aa9a1ba3f23e6428b5 (DEFAULT_TABLE table=default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.830843 4192 tablet_service.cc:1511] Processing CreateTablet for tablet e9927db780674f31be8f043fe0843736 (DEFAULT_TABLE table=default.Table_8ebf726da3884df1b2b6805cb6b147a5 [id=54576f526a0c449e90b1e8c86326b803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.841786 4654 ts_tablet_manager.cc:1916] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:51.841853 4911 raft_consensus.cc:2804] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:51.842147 4911 raft_consensus.cc:493] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:51.842438 4911 raft_consensus.cc:3060] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.848975 4530 raft_consensus.cc:2468] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:51.848856 5052 tablet.cc:1724] T 9d1b860c7073446597ce696f822ce1c2 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:51.851881 4158 leader_election.cc:304] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:51.852237 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12d0f56f8db41aa9a1ba3f23e6428b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.852615 4204 tablet_service.cc:1511] Processing CreateTablet for tablet bdf98515d8284e3b9b84480807e8108a (DEFAULT_TABLE table=default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.853919 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf98515d8284e3b9b84480807e8108a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.854247 5032 raft_consensus.cc:2749] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:51.854657 4363 tablet_service.cc:1511] Processing CreateTablet for tablet e12d0f56f8db41aa9a1ba3f23e6428b5 (DEFAULT_TABLE table=default.Table_7f29ad8fb22d4a378afb83449fab451b [id=b75f16c3c90c4b6bb561726761e3077d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.855157 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9927db780674f31be8f043fe0843736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.855590 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12d0f56f8db41aa9a1ba3f23e6428b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.866107 4503 tablet_service.cc:1511] Processing CreateTablet for tablet a9ee8deb609742a3ae6f4f1c0b13308f (DEFAULT_TABLE table=default.Table_79d9a94d02f94e8baa483828c8218407 [id=18ce412ad9b0431f8a2e63ec9a313118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.867169 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 958cf4732e254c75bda36a6a550b93ef (DEFAULT_TABLE table=default.Table_a7ec0d6efc404f0d80ccdee5fc87c213 [id=12bfce5ea1a344748499a5ec833a2ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.867210 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9ee8deb609742a3ae6f4f1c0b13308f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.868608 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 2c119dd9a5d944eabd588f62a76da98e (table default.Table_445a32d80e864772b0a615ef426ccadb [id=51ad748be7c14d32ac99d6f8f63f569e]) successfully deleted
I20260430 19:58:51.870990 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958cf4732e254c75bda36a6a550b93ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.877518 4911 raft_consensus.cc:515] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.879331 4911 leader_election.cc:290] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:51.884280 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09f272340cd942e9b2af9e38a2bdec82" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:51.884846 4222 raft_consensus.cc:3060] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:51.891700 4222 raft_consensus.cc:2468] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:51.893567 4470 leader_election.cc:304] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:51.895074 4968 raft_consensus.cc:2804] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:51.892830 4818 tablet_bootstrap.cc:492] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:51.895490 4968 raft_consensus.cc:697] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:51.895961 4818 ts_tablet_manager.cc:1403] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.524s user 0.010s sys 0.000s
I20260430 19:58:51.896757 4968 consensus_queue.cc:237] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.897596 4654 ts_tablet_manager.cc:1929] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:51.897943 4654 log.cc:1199] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/404f20adf9d741f3a036b06cbdbaedb8
I20260430 19:58:51.898865 5032 raft_consensus.cc:493] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:51.899277 5032 raft_consensus.cc:515] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:51.899566 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ba1d91aaad6e4d3db7fdae67076dd6c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:51.901058 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba1d91aaad6e4d3db7fdae67076dd6c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.902336 5032 leader_election.cc:290] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:51.903218 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_95badffa04c845919ea1d575c4df3244"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:51.905025 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95badffa04c845919ea1d575c4df3244 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:51.908418 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c81b11280f6d49e681537c88fb1cbf68" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:51.911238 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c81b11280f6d49e681537c88fb1cbf68" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:51.917232 4505 tablet_service.cc:1511] Processing CreateTablet for tablet f50116c63b0747099de2d234ca3faaac (DEFAULT_TABLE table=default.Table_830b99b7bf1e49e8ad90881ae777dca7 [id=dfb715d58ee24771a11cb779aa26c4c6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:51.919025 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 404f20adf9d741f3a036b06cbdbaedb8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 404f20adf9d741f3a036b06cbdbaedb8 already in progress: creating tablet
I20260430 19:58:51.921552 4354 tablet_service.cc:1511] Processing CreateTablet for tablet a9ee8deb609742a3ae6f4f1c0b13308f (DEFAULT_TABLE table=default.Table_79d9a94d02f94e8baa483828c8218407 [id=18ce412ad9b0431f8a2e63ec9a313118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.922677 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9ee8deb609742a3ae6f4f1c0b13308f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.923300 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 958cf4732e254c75bda36a6a550b93ef (DEFAULT_TABLE table=default.Table_a7ec0d6efc404f0d80ccdee5fc87c213 [id=12bfce5ea1a344748499a5ec833a2ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.923837 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09f272340cd942e9b2af9e38a2bdec82" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:51.924345 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958cf4732e254c75bda36a6a550b93ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:51.924955 4470 leader_election.cc:343] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:51.936165 4968 consensus_queue.cc:1048] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:51.936995 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50116c63b0747099de2d234ca3faaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.945497 4729 tablet_bootstrap.cc:492] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:51.898555 4654 ts_tablet_manager.cc:1950] T 404f20adf9d741f3a036b06cbdbaedb8 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:51.954921 4729 ts_tablet_manager.cc:1403] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.241s user 0.011s sys 0.008s
I20260430 19:58:51.967361 4354 tablet_service.cc:1511] Processing CreateTablet for tablet f50116c63b0747099de2d234ca3faaac (DEFAULT_TABLE table=default.Table_830b99b7bf1e49e8ad90881ae777dca7 [id=dfb715d58ee24771a11cb779aa26c4c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.968550 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50116c63b0747099de2d234ca3faaac. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:51.980331 4161 leader_election.cc:343] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:51.983904 4158 leader_election.cc:343] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:51.993903 4158 leader_election.cc:304] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:51.994800 4960 raft_consensus.cc:2749] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:51.998772 4190 tablet_service.cc:1511] Processing CreateTablet for tablet a9ee8deb609742a3ae6f4f1c0b13308f (DEFAULT_TABLE table=default.Table_79d9a94d02f94e8baa483828c8218407 [id=18ce412ad9b0431f8a2e63ec9a313118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:51.999802 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9ee8deb609742a3ae6f4f1c0b13308f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:51.999722 4193 tablet_service.cc:1511] Processing CreateTablet for tablet f50116c63b0747099de2d234ca3faaac (DEFAULT_TABLE table=default.Table_830b99b7bf1e49e8ad90881ae777dca7 [id=dfb715d58ee24771a11cb779aa26c4c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.000311 4503 tablet_service.cc:1511] Processing CreateTablet for tablet cfd78a1cd7a94e27bb376d1b7c151073 (DEFAULT_TABLE table=default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.000730 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50116c63b0747099de2d234ca3faaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.001153 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 958cf4732e254c75bda36a6a550b93ef (DEFAULT_TABLE table=default.Table_a7ec0d6efc404f0d80ccdee5fc87c213 [id=12bfce5ea1a344748499a5ec833a2ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.001346 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd78a1cd7a94e27bb376d1b7c151073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.001991 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958cf4732e254c75bda36a6a550b93ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.006794 4776 tablet_bootstrap.cc:654] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.002781 4818 raft_consensus.cc:359] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.009007 4818 raft_consensus.cc:385] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.009261 4818 raft_consensus.cc:740] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.015818 4358 tablet_service.cc:1511] Processing CreateTablet for tablet cfd78a1cd7a94e27bb376d1b7c151073 (DEFAULT_TABLE table=default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.016680 4203 tablet_service.cc:1511] Processing CreateTablet for tablet cfd78a1cd7a94e27bb376d1b7c151073 (DEFAULT_TABLE table=default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.017589 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd78a1cd7a94e27bb376d1b7c151073. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:52.018684 4470 consensus_peers.cc:597] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:52.020597 4776 tablet_bootstrap.cc:492] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.020984 4776 ts_tablet_manager.cc:1403] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.205s user 0.003s sys 0.006s
I20260430 19:58:52.022907 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 404f20adf9d741f3a036b06cbdbaedb8 (table default.Table_2afe6c2b4071491f868a3a17cc4d6e48 [id=06859fba4a5d4b8f978e6f56edc5fb85]) successfully deleted
I20260430 19:58:52.023137 4776 raft_consensus.cc:359] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.023728 4776 raft_consensus.cc:385] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.023970 4776 raft_consensus.cc:740] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.024667 4776 consensus_queue.cc:260] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.028614 4776 ts_tablet_manager.cc:1434] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260430 19:58:52.017570 4818 consensus_queue.cc:260] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.031241 4776 tablet_bootstrap.cc:492] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.036114 4818 ts_tablet_manager.cc:1434] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.140s user 0.005s sys 0.000s
I20260430 19:58:52.056574 4729 raft_consensus.cc:359] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.057331 4729 raft_consensus.cc:385] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.057634 4729 raft_consensus.cc:740] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.063616 4729 consensus_queue.cc:260] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.062420 5059 tablet.cc:1724] T 9d1b860c7073446597ce696f822ce1c2 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:52.074291 4776 tablet_bootstrap.cc:654] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.074975 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd78a1cd7a94e27bb376d1b7c151073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.106452 4729 ts_tablet_manager.cc:1434] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.144s user 0.005s sys 0.003s
I20260430 19:58:52.107605 4818 tablet_bootstrap.cc:492] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:52.113027 4776 tablet_bootstrap.cc:492] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.113575 4776 ts_tablet_manager.cc:1403] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.084s user 0.005s sys 0.005s
I20260430 19:58:52.116442 4776 raft_consensus.cc:359] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.117090 4776 raft_consensus.cc:385] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.117340 4776 raft_consensus.cc:740] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.118189 4776 consensus_queue.cc:260] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.120352 4776 ts_tablet_manager.cc:1434] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260430 19:58:52.123406 4776 tablet_bootstrap.cc:492] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.128216 4021 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 443308f2a26f4622aa4551ff5f73fc39
I20260430 19:58:52.129585 4021 catalog_manager.cc:6741] ReplaceTablet: tablet 443308f2a26f4622aa4551ff5f73fc39 deleted and replaced by tablet 70e4d5af75d04f6d9091d2b086e4a6ba
I20260430 19:58:52.130160 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet 3423379d1e3d4020a4241bf8917f1050 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:52.136499 4776 tablet_bootstrap.cc:654] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.146296 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_db210d74dff541908dd0daed2a35c325"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.147918 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db210d74dff541908dd0daed2a35c325 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:52.150707 4776 tablet_bootstrap.cc:492] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.151072 4776 ts_tablet_manager.cc:1403] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.004s
W20260430 19:58:52.150456 4023 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.2:49905 (request call id 40) took 840 ms (client timeout 1000 ms). Trace:
W20260430 19:58:52.154697 4023 rpcz_store.cc:269] 0430 19:58:51.310420 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:51.314677 (+ 4257us) service_pool.cc:225] Handling call
0430 19:58:52.150401 (+835724us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6}
I20260430 19:58:52.155957 4023 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_9312a99c3d0040e3a6c62e6400b06041" } modify_external_catalogs: true
I20260430 19:58:52.156445 4023 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_9312a99c3d0040e3a6c62e6400b06041" } modify_external_catalogs: true
I20260430 19:58:52.156790 4776 raft_consensus.cc:359] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.157382 4776 raft_consensus.cc:385] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.157620 4776 raft_consensus.cc:740] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.159776 4776 consensus_queue.cc:260] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.163286 4776 ts_tablet_manager.cc:1434] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260430 19:58:52.164222 4776 tablet_bootstrap.cc:492] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.176780 4776 tablet_bootstrap.cc:654] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.176690 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 with delete_type TABLET_DATA_DELETED (replaced by tablet 70e4d5af75d04f6d9091d2b086e4a6ba) from {username='slave'} at 127.0.0.1:41996
W20260430 19:58:52.179987 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 443308f2a26f4622aa4551ff5f73fc39 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 443308f2a26f4622aa4551ff5f73fc39 already in progress: creating tablet
I20260430 19:58:52.181728 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 with delete_type TABLET_DATA_DELETED (replaced by tablet 70e4d5af75d04f6d9091d2b086e4a6ba) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:52.136462 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 with delete_type TABLET_DATA_DELETED (replaced by tablet 70e4d5af75d04f6d9091d2b086e4a6ba) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:52.185842 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 443308f2a26f4622aa4551ff5f73fc39 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 443308f2a26f4622aa4551ff5f73fc39 already in progress: creating tablet
W20260430 19:58:52.189916 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 443308f2a26f4622aa4551ff5f73fc39 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 443308f2a26f4622aa4551ff5f73fc39 already in progress: creating tablet
I20260430 19:58:52.192643 4776 tablet_bootstrap.cc:492] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.193910 4776 ts_tablet_manager.cc:1403] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.006s
I20260430 19:58:52.188349 4818 tablet_bootstrap.cc:654] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.198989 4776 raft_consensus.cc:359] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.199589 4776 raft_consensus.cc:385] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.194506 4729 tablet_bootstrap.cc:492] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:52.199846 4776 raft_consensus.cc:740] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.201846 4776 consensus_queue.cc:260] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.200275 4911 consensus_queue.cc:1048] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:52.205346 4776 ts_tablet_manager.cc:1434] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260430 19:58:52.206198 4776 tablet_bootstrap.cc:492] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.222082 4776 tablet_bootstrap.cc:654] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.229732 4729 tablet_bootstrap.cc:654] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.234253 4776 tablet_bootstrap.cc:492] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.234683 4776 ts_tablet_manager.cc:1403] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.004s
I20260430 19:58:52.236953 4776 raft_consensus.cc:359] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.237541 4776 raft_consensus.cc:385] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.237785 4776 raft_consensus.cc:740] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.243534 5034 tablet.cc:1724] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:52.242993 4776 consensus_queue.cc:260] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.246470 5052 tablet.cc:1724] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:52.250466 4818 tablet_bootstrap.cc:492] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:52.250972 4818 ts_tablet_manager.cc:1403] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.144s user 0.008s sys 0.002s
I20260430 19:58:52.232308 5059 tablet.cc:1724] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:52.258795 4776 ts_tablet_manager.cc:1434] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.024s user 0.003s sys 0.002s
W20260430 19:58:52.260422 4026 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 958cf4732e254c75bda36a6a550b93ef consensus state has no leader
I20260430 19:58:52.261449 4024 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:52.263288 5078 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.62:33781 every 8 connection(s)
I20260430 19:58:52.263855 4635 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.62:33781
I20260430 19:58:52.268015 4635 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
I20260430 19:58:52.274816 4193 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:52.275007 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 4635
I20260430 19:58:52.275079 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet c81b11280f6d49e681537c88fb1cbf68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:52.279379 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet c81b11280f6d49e681537c88fb1cbf68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:52.279428 4029 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.3:35079 (request call id 36) took 869 ms (client timeout 1000 ms). Trace:
I20260430 19:58:52.280061 4915 tablet_replica.cc:333] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:52.280764 4915 raft_consensus.cc:2243] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:52.281253 4915 raft_consensus.cc:2272] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:52.283378 4915 ts_tablet_manager.cc:1916] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:52.284221 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e8206ac4484abfa769276c83fd3333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:52.284868 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:52.279908 4029 rpcz_store.cc:269] 0430 19:58:51.410515 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:51.431058 (+ 20543us) service_pool.cc:225] Handling call
0430 19:58:52.279370 (+848312us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:52.037742 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:52.038137 (+ 395us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:52.038165 (+ 28us) write_op.cc:435] Acquired schema lock
0430 19:58:52.038177 (+ 12us) tablet.cc:662] Decoding operations
0430 19:58:52.038489 (+ 312us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:52.038524 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:52.038550 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0430 19:58:52.038845 (+ 295us) tablet.cc:701] Row locks acquired
0430 19:58:52.038858 (+ 13us) write_op.cc:260] PREPARE: finished
0430 19:58:52.038970 (+ 112us) write_op.cc:270] Start()
0430 19:58:52.039054 (+ 84us) write_op.cc:276] Timestamp: P: 1777579132038869 usec, L: 0
0430 19:58:52.039069 (+ 15us) op_driver.cc:348] REPLICATION: starting
0430 19:58:52.039547 (+ 478us) log.cc:844] Serialized 901 byte log entry
0430 19:58:52.251475 (+211928us) op_driver.cc:464] REPLICATION: finished
0430 19:58:52.256805 (+ 5330us) write_op.cc:301] APPLY: starting
0430 19:58:52.256849 (+ 44us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:52.256938 (+ 89us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:52.257009 (+ 71us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:52.257196 (+ 187us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:52.257311 (+ 115us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0430 19:58:52.257347 (+ 36us) write_op.cc:312] APPLY: finished
0430 19:58:52.257487 (+ 140us) log.cc:844] Serialized 29 byte log entry
0430 19:58:52.257609 (+ 122us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:52.257658 (+ 49us) write_op.cc:454] Released schema lock
0430 19:58:52.257683 (+ 25us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":5201,"apply.run_cpu_time_us":1083,"apply.run_wall_time_us":12769,"num_ops":2,"prepare.queue_time_us":273373,"prepare.run_cpu_time_us":2074,"prepare.run_wall_time_us":24312,"replication_time_us":212332,"threads_started":1}]]}
I20260430 19:58:52.285482 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c119dd9a5d944eabd588f62a76da98e with delete_type TABLET_DATA_DELETED (replaced by tablet 239a2c6a534f4f478e436d14f94cd762) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:52.286386 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet bdf98515d8284e3b9b84480807e8108a) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:52.287235 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet c81b11280f6d49e681537c88fb1cbf68 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c81b11280f6d49e681537c88fb1cbf68 already in progress: creating tablet
W20260430 19:58:52.266476 4028 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.1:59387 (request call id 47) took 993 ms (client timeout 1000 ms). Trace:
I20260430 19:58:52.295585 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet c81b11280f6d49e681537c88fb1cbf68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:52.299697 4776 tablet_bootstrap.cc:492] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.303206 4742 tablet_replica.cc:333] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:52.303917 4742 raft_consensus.cc:2243] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:52.304082 4915 ts_tablet_manager.cc:1929] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:52.304330 4742 raft_consensus.cc:2272] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:52.304414 4915 log.cc:1199] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:52.304986 4915 ts_tablet_manager.cc:1950] T c81b11280f6d49e681537c88fb1cbf68 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:52.308429 4776 tablet_bootstrap.cc:654] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.309152 4742 ts_tablet_manager.cc:1916] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:52.296159 4028 rpcz_store.cc:269] 0430 19:58:51.273146 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:51.273258 (+ 112us) service_pool.cc:225] Handling call
0430 19:58:51.310368 (+ 37110us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:52.266413 (+956045us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:51.553209 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:51.553556 (+ 347us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:51.553581 (+ 25us) write_op.cc:435] Acquired schema lock
0430 19:58:51.553595 (+ 14us) tablet.cc:662] Decoding operations
0430 19:58:51.553872 (+ 277us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:51.553909 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:51.553933 (+ 24us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:51.554067 (+ 134us) tablet.cc:701] Row locks acquired
0430 19:58:51.554081 (+ 14us) write_op.cc:260] PREPARE: finished
0430 19:58:51.554182 (+ 101us) write_op.cc:270] Start()
0430 19:58:51.554258 (+ 76us) write_op.cc:276] Timestamp: P: 1777579131554086 usec, L: 0
0430 19:58:51.554274 (+ 16us) op_driver.cc:348] REPLICATION: starting
0430 19:58:51.554763 (+ 489us) log.cc:844] Serialized 356 byte log entry
0430 19:58:51.778906 (+224143us) op_driver.cc:464] REPLICATION: finished
0430 19:58:51.782612 (+ 3706us) write_op.cc:301] APPLY: starting
0430 19:58:51.785230 (+ 2618us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:51.785322 (+ 92us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:51.785431 (+ 109us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:51.785613 (+ 182us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:51.785761 (+ 148us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:51.785807 (+ 46us) write_op.cc:312] APPLY: finished
0430 19:58:51.785971 (+ 164us) log.cc:844] Serialized 23 byte log entry
0430 19:58:51.786113 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:51.786174 (+ 61us) write_op.cc:454] Released schema lock
0430 19:58:51.786206 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":7288,"reported_tablets":6,"spinlock_wait_cycles":81685376,"child_traces":[["op",{"apply.queue_time_us":3559,"apply.run_cpu_time_us":1288,"apply.run_wall_time_us":3808,"num_ops":1,"prepare.queue_time_us":79779,"prepare.run_cpu_time_us":1839,"prepare.run_wall_time_us":159130,"raft.queue_time_us":129071,"raft.run_cpu_time_us":18,"raft.run_wall_time_us":19,"replication_time_us":224560,"thread_start_us":119622,"threads_started":1}]]}
I20260430 19:58:52.315347 4912 consensus_queue.cc:1048] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:52.254247 4818 raft_consensus.cc:359] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.326695 4818 raft_consensus.cc:385] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.331271 4818 raft_consensus.cc:740] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.332921 4818 consensus_queue.cc:260] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.345821 4818 ts_tablet_manager.cc:1434] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.095s user 0.006s sys 0.000s
I20260430 19:58:52.346717 4776 tablet_bootstrap.cc:492] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.347105 4776 ts_tablet_manager.cc:1403] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.002s
I20260430 19:58:52.348634 4729 tablet_bootstrap.cc:492] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:52.349148 4729 ts_tablet_manager.cc:1403] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.156s user 0.005s sys 0.005s
I20260430 19:58:52.351599 4729 raft_consensus.cc:359] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.352847 4818 tablet_bootstrap.cc:492] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:52.353624 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet c81b11280f6d49e681537c88fb1cbf68 (table default.Table_f79093a8060a4af3979bd0705d8a6998 [id=7b43b6a692214e0b89c73b9374682197]) successfully deleted
I20260430 19:58:52.354856 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_d13179c843c54969b98111d67f829f5f" } new_table_name: "default.Table_60d3a33c8bfc4641ac60f0ea151c1225" modify_external_catalogs: true
W20260430 19:58:52.355049 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 13e8206ac4484abfa769276c83fd3333 because the tablet was not found. No further retry: Not found: Tablet not found: 13e8206ac4484abfa769276c83fd3333
I20260430 19:58:52.355696 4729 raft_consensus.cc:385] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.356003 4729 raft_consensus.cc:740] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.356600 4915 tablet_replica.cc:333] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:52.366374 4915 raft_consensus.cc:2243] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:52.372045 4915 pending_rounds.cc:70] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Trying to abort 1 pending ops.
I20260430 19:58:52.372380 4915 pending_rounds.cc:77] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964124262264832 op_type: NO_OP noop_request { }
I20260430 19:58:52.365448 4729 consensus_queue.cc:260] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.374291 4729 ts_tablet_manager.cc:1434] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.025s user 0.004s sys 0.000s
I20260430 19:58:52.375172 4021 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 3423379d1e3d4020a4241bf8917f1050
I20260430 19:58:52.376391 4021 catalog_manager.cc:6741] ReplaceTablet: tablet 3423379d1e3d4020a4241bf8917f1050 deleted and replaced by tablet 6f0489b4631346f4952ef648e0653242
I20260430 19:58:52.376987 4776 raft_consensus.cc:359] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.377646 4776 raft_consensus.cc:385] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.378973 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 3423379d1e3d4020a4241bf8917f1050 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f0489b4631346f4952ef648e0653242) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:52.381385 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 3423379d1e3d4020a4241bf8917f1050 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f0489b4631346f4952ef648e0653242) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:52.382290 4023 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 50d8b9a982fe4810858d1ec2b28d7894
W20260430 19:58:52.386435 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 3423379d1e3d4020a4241bf8917f1050 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3423379d1e3d4020a4241bf8917f1050 already in progress: creating tablet
I20260430 19:58:52.392985 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 50d8b9a982fe4810858d1ec2b28d7894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:52.394104 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 50d8b9a982fe4810858d1ec2b28d7894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:52.395586 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 50d8b9a982fe4810858d1ec2b28d7894 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50d8b9a982fe4810858d1ec2b28d7894 already in progress: creating tablet
I20260430 19:58:52.377892 4776 raft_consensus.cc:740] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.399748 4776 consensus_queue.cc:260] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.385510 4514 tablet_service.cc:1558] Processing DeleteTablet for tablet 3423379d1e3d4020a4241bf8917f1050 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f0489b4631346f4952ef648e0653242) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:52.407538 4776 ts_tablet_manager.cc:1434] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.060s user 0.004s sys 0.001s
I20260430 19:58:52.390462 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet 50d8b9a982fe4810858d1ec2b28d7894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:52.372840 4915 raft_consensus.cc:2889] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:52.408257 4915 raft_consensus.cc:2272] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:52.408497 4776 tablet_bootstrap.cc:492] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.410769 4915 ts_tablet_manager.cc:1916] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:52.421159 4028 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f0489b4631346f4952ef648e0653242 has no consensus state
I20260430 19:58:52.427043 4742 ts_tablet_manager.cc:1929] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:52.431124 4729 tablet_bootstrap.cc:492] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:52.432750 4028 master_service.cc:954] ReplaceTablet: received request to replace tablet b6acec54bc7b4692894333e0150a1aff from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:52.435364 4742 log.cc:1199] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/09f272340cd942e9b2af9e38a2bdec82
I20260430 19:58:52.436496 4818 tablet_bootstrap.cc:654] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.442941 4581 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:52.443454 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:52.450678 4742 ts_tablet_manager.cc:1950] T 09f272340cd942e9b2af9e38a2bdec82 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:52.453287 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 09f272340cd942e9b2af9e38a2bdec82 (table default.Table_276bd93958054b6b9ad30f314bbb487c [id=de548c1ed3d8424b9a2d3c451444462c]) successfully deleted
I20260430 19:58:52.453764 4729 tablet_bootstrap.cc:654] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.454243 4742 tablet_replica.cc:333] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:52.455708 4960 raft_consensus.cc:493] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.456151 4960 raft_consensus.cc:515] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.457996 4960 leader_election.cc:290] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:52.459443 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e34f3b1bf84f72855cf87683c5e318" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:52.459789 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4e34f3b1bf84f72855cf87683c5e318" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:52.461908 4729 tablet_bootstrap.cc:492] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:52.463304 4161 leader_election.cc:304] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.464558 5014 raft_consensus.cc:493] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.465039 4960 raft_consensus.cc:2749] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.465018 5014 raft_consensus.cc:515] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.466818 5014 leader_election.cc:290] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
W20260430 19:58:52.468964 4320 leader_election.cc:343] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: 2c119dd9a5d944eabd588f62a76da98e
I20260430 19:58:52.469609 4729 ts_tablet_manager.cc:1403] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.006s
I20260430 19:58:52.472357 4729 raft_consensus.cc:359] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.473774 4023 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_79d9a94d02f94e8baa483828c8218407" } new_table_name: "default.Table_5be52b4f74d74f8ca79ba986a2ab7334" modify_external_catalogs: true
I20260430 19:58:52.474875 4776 tablet_bootstrap.cc:654] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.475001 4027 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_f01cb568b13b4e5ab73b36deda0357f3" } modify_external_catalogs: true
I20260430 19:58:52.475452 4027 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_f01cb568b13b4e5ab73b36deda0357f3" } modify_external_catalogs: true
I20260430 19:58:52.476872 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_1a0b8c0b26af42519951ec338ca684f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.478245 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a0b8c0b26af42519951ec338ca684f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:52.481928 4317 leader_election.cc:343] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 2c119dd9a5d944eabd588f62a76da98e
I20260430 19:58:52.482331 4317 leader_election.cc:304] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:52.484457 5014 raft_consensus.cc:2749] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.484954 4274 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:52.488436 4436 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:52.467741 4742 raft_consensus.cc:2243] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:52.493686 4729 raft_consensus.cc:385] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.494834 5079 sys_catalog.cc:263] Verifying existing consensus state
I20260430 19:58:52.495379 4818 tablet_bootstrap.cc:492] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:52.495788 4818 ts_tablet_manager.cc:1403] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.143s user 0.015s sys 0.000s
I20260430 19:58:52.498133 4818 raft_consensus.cc:359] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.500708 4776 tablet_bootstrap.cc:492] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.501046 4222 raft_consensus.cc:3060] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:52.501121 4776 ts_tablet_manager.cc:1403] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.093s user 0.008s sys 0.001s
I20260430 19:58:52.503376 4742 raft_consensus.cc:2272] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:52.505015 4742 ts_tablet_manager.cc:1916] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:52.504972 4776 raft_consensus.cc:359] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.505600 4776 raft_consensus.cc:385] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.505836 4776 raft_consensus.cc:740] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.493973 4729 raft_consensus.cc:740] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.508744 4729 consensus_queue.cc:260] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.510659 4729 ts_tablet_manager.cc:1434] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260430 19:58:52.511538 4729 tablet_bootstrap.cc:492] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:52.513734 4776 consensus_queue.cc:260] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.518990 4776 ts_tablet_manager.cc:1434] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260430 19:58:52.519807 4776 tablet_bootstrap.cc:492] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
W20260430 19:58:52.521786 4023 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a9ee8deb609742a3ae6f4f1c0b13308f consensus state has no leader
I20260430 19:58:52.531523 4915 ts_tablet_manager.cc:1929] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:52.531970 4818 raft_consensus.cc:385] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.532335 4818 raft_consensus.cc:740] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.533068 4818 consensus_queue.cc:260] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.537433 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:52.533983 4915 log.cc:1199] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/09f272340cd942e9b2af9e38a2bdec82
I20260430 19:58:52.538436 4915 ts_tablet_manager.cc:1950] T 09f272340cd942e9b2af9e38a2bdec82 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:52.541079 5079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Bootstrap starting.
I20260430 19:58:52.542671 5014 raft_consensus.cc:493] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.543102 5014 raft_consensus.cc:515] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.545392 5014 leader_election.cc:290] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:52.548184 4960 raft_consensus.cc:493] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260430 19:58:52.548825 4317 leader_election.cc:343] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: f3db0439f4f042a4a9cb38ebd3764ab5
I20260430 19:58:52.548609 4960 raft_consensus.cc:515] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:52.555759 4320 leader_election.cc:343] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: f3db0439f4f042a4a9cb38ebd3764ab5
I20260430 19:58:52.556138 4320 leader_election.cc:304] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:52.559345 5014 raft_consensus.cc:2749] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.560621 4960 leader_election.cc:290] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:52.562300 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d410f3b3b04e649b79dda04c0891ee" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:52.563606 4161 leader_election.cc:343] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.567318 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_4472ad7bad434d159c5275dc870c286c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:52.568054 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a9a78f62539447b98602897b63dd48f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.568775 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4472ad7bad434d159c5275dc870c286c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:52.569402 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9a78f62539447b98602897b63dd48f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:52.571009 4776 tablet_bootstrap.cc:654] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.576723 4818 ts_tablet_manager.cc:1434] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.081s user 0.006s sys 0.000s
I20260430 19:58:52.559763 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d410f3b3b04e649b79dda04c0891ee" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:52.580905 4158 leader_election.cc:343] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.581290 4158 leader_election.cc:304] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.582250 4960 raft_consensus.cc:2749] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.532698 4729 tablet_bootstrap.cc:654] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.590663 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_149dd50933e148538c23f32cf19ca2dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.592229 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_149dd50933e148538c23f32cf19ca2dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:52.597273 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5cbfe354833844588121cc44d0a76e64" } modify_external_catalogs: true
I20260430 19:58:52.597790 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5cbfe354833844588121cc44d0a76e64" } modify_external_catalogs: true
W20260430 19:58:52.602936 4320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 47200.3us
I20260430 19:58:52.606652 5093 raft_consensus.cc:493] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.607376 5093 raft_consensus.cc:515] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.610952 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745f04d04091443ca1c39949d005dd69" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:52.611296 4014 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet cfd78a1cd7a94e27bb376d1b7c151073 (table default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cfd78a1cd7a94e27bb376d1b7c151073 consensus state has no leader
I20260430 19:58:52.611545 4530 raft_consensus.cc:2468] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:52.613039 4158 leader_election.cc:304] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:52.614018 4818 tablet_bootstrap.cc:492] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:52.614135 4960 raft_consensus.cc:2804] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:52.614441 4960 raft_consensus.cc:493] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:52.614774 4960 raft_consensus.cc:3060] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:52.619699 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 09f272340cd942e9b2af9e38a2bdec82 (table default.Table_276bd93958054b6b9ad30f314bbb487c [id=de548c1ed3d8424b9a2d3c451444462c]) successfully deleted
W20260430 19:58:52.621153 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 2c119dd9a5d944eabd588f62a76da98e because the tablet was not found. No further retry: Not found: Tablet not found: 2c119dd9a5d944eabd588f62a76da98e
I20260430 19:58:52.622123 4915 tablet_replica.cc:333] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:52.623349 4776 tablet_bootstrap.cc:492] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.623808 4776 ts_tablet_manager.cc:1403] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.104s user 0.008s sys 0.002s
I20260430 19:58:52.623428 4960 raft_consensus.cc:515] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.624933 4021 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_445a32d80e864772b0a615ef426ccadb" } new_table_name: "default.Table_cf1378d9973a48079c8e2b0bea84dc2a" modify_external_catalogs: true
I20260430 19:58:52.626470 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745f04d04091443ca1c39949d005dd69" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:52.627082 4530 raft_consensus.cc:3060] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:52.634930 4915 raft_consensus.cc:2243] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:52.635399 4915 raft_consensus.cc:2272] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:52.635911 4530 raft_consensus.cc:2468] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:52.636997 4158 leader_election.cc:304] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:52.641309 4776 raft_consensus.cc:359] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.641939 4776 raft_consensus.cc:385] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.642235 4776 raft_consensus.cc:740] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.643579 4729 tablet_bootstrap.cc:492] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:52.645864 4729 ts_tablet_manager.cc:1403] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.135s user 0.008s sys 0.002s
I20260430 19:58:52.648236 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745f04d04091443ca1c39949d005dd69" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
W20260430 19:58:52.649518 4161 leader_election.cc:343] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.650072 4915 ts_tablet_manager.cc:1916] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:52.649806 4776 consensus_queue.cc:260] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.651751 4960 leader_election.cc:290] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:52.652925 4960 raft_consensus.cc:2804] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:52.653348 4960 raft_consensus.cc:697] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:52.661207 4776 ts_tablet_manager.cc:1434] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.037s user 0.006s sys 0.001s
I20260430 19:58:52.662130 4776 tablet_bootstrap.cc:492] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.664394 4818 tablet_bootstrap.cc:654] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.668185 5098 raft_consensus.cc:493] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.669148 5079 log.cc:826] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Log is configured to *not* fsync() on all Append() calls
I20260430 19:58:52.668646 5098 raft_consensus.cc:515] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.671726 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "654df09f602b4f96bcf5d13a514df55c" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:52.675482 5098 leader_election.cc:290] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:52.678411 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "654df09f602b4f96bcf5d13a514df55c" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:52.678706 4776 tablet_bootstrap.cc:654] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.679925 4158 leader_election.cc:304] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.680902 5098 raft_consensus.cc:2749] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.685310 5098 raft_consensus.cc:493] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.685822 5098 raft_consensus.cc:515] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:52.689020 4742 ts_tablet_manager.cc:1929] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260430 19:58:52.689185 4161 leader_election.cc:343] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Not found: Tablet not found: 404f20adf9d741f3a036b06cbdbaedb8
I20260430 19:58:52.689523 4742 log.cc:1199] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:52.689982 5098 leader_election.cc:290] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:52.690217 4742 ts_tablet_manager.cc:1950] T c81b11280f6d49e681537c88fb1cbf68 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
W20260430 19:58:52.694227 4158 leader_election.cc:343] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 404f20adf9d741f3a036b06cbdbaedb8
I20260430 19:58:52.694780 4158 leader_election.cc:304] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.695825 5097 raft_consensus.cc:2749] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.697676 4776 tablet_bootstrap.cc:492] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:52.697750 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745f04d04091443ca1c39949d005dd69" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:52.698057 4776 ts_tablet_manager.cc:1403] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.003s
W20260430 19:58:52.698961 4161 leader_election.cc:343] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.699460 5093 leader_election.cc:290] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:52.700182 4818 tablet_bootstrap.cc:492] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:52.700588 4818 ts_tablet_manager.cc:1403] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.087s user 0.010s sys 0.000s
I20260430 19:58:52.701596 5079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260430 19:58:52.701790 4729 raft_consensus.cc:359] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.702505 4729 raft_consensus.cc:385] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.702571 5079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Bootstrap complete.
I20260430 19:58:52.702981 4818 raft_consensus.cc:359] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.654168 4960 consensus_queue.cc:237] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.710029 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a2d74c305d5f4d8ca3db561150380152"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:52.691687 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet c81b11280f6d49e681537c88fb1cbf68 (table default.Table_f79093a8060a4af3979bd0705d8a6998 [id=7b43b6a692214e0b89c73b9374682197]) successfully deleted
W20260430 19:58:52.712092 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 3423379d1e3d4020a4241bf8917f1050 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3423379d1e3d4020a4241bf8917f1050 already in progress: creating tablet
W20260430 19:58:52.712670 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 50d8b9a982fe4810858d1ec2b28d7894 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50d8b9a982fe4810858d1ec2b28d7894 already in progress: creating tablet
I20260430 19:58:52.718722 4818 raft_consensus.cc:385] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.719180 4818 raft_consensus.cc:740] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.719920 4818 consensus_queue.cc:260] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.725003 4023 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_58ec6ea9c8dc4e2c9d35ab8648add502" } new_table_name: "default.Table_41ec2a62305f427a9f2a581b7dce9c85" modify_external_catalogs: true
I20260430 19:58:52.735548 4729 raft_consensus.cc:740] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.737391 4729 consensus_queue.cc:260] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.743371 4776 raft_consensus.cc:359] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.744041 4776 raft_consensus.cc:385] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.744288 4776 raft_consensus.cc:740] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.744844 4776 consensus_queue.cc:260] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.746028 5014 consensus_queue.cc:1048] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:52.746750 4776 ts_tablet_manager.cc:1434] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
I20260430 19:58:52.747530 4776 tablet_bootstrap.cc:492] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:52.751230 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
W20260430 19:58:52.756517 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2d74c305d5f4d8ca3db561150380152 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:52.757062 4729 ts_tablet_manager.cc:1434] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.110s user 0.006s sys 0.000s
I20260430 19:58:52.759538 4729 tablet_bootstrap.cc:492] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:52.762850 4776 tablet_bootstrap.cc:654] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.764389 4818 ts_tablet_manager.cc:1434] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.064s user 0.005s sys 0.000s
I20260430 19:58:52.770810 5098 raft_consensus.cc:493] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.771255 5098 raft_consensus.cc:515] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.773705 4818 tablet_bootstrap.cc:492] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:52.774081 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e62335e43c1410b8b3acdd9e2f47981" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:52.774273 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet b6acec54bc7b4692894333e0150a1aff from {username='slave'} at 127.0.0.1:55130
W20260430 19:58:52.775311 4161 leader_election.cc:343] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.775908 5098 leader_election.cc:290] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:52.777498 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e62335e43c1410b8b3acdd9e2f47981" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:52.778942 4158 leader_election.cc:343] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.779312 4158 leader_election.cc:304] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.780174 5098 raft_consensus.cc:2749] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.782261 4915 ts_tablet_manager.cc:1929] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:52.782624 4915 log.cc:1199] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/37d6c54ae89946158cad45e741c1c0f0
I20260430 19:58:52.783197 4915 ts_tablet_manager.cc:1950] T 37d6c54ae89946158cad45e741c1c0f0 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:52.780977 5079 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:52.784265 5079 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0a440dabed4814aaefe431944a7b5e, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.784307 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 37d6c54ae89946158cad45e741c1c0f0 (table default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]) successfully deleted
I20260430 19:58:52.785172 5079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:52.794767 5014 consensus_queue.cc:1048] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:52.799180 5107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:52.800091 5107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:52.802901 5079 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: configured and running, proceeding with master startup.
I20260430 19:58:52.808033 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_4ee70552d7224ff9a90e149ed7a17de4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.809680 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ee70552d7224ff9a90e149ed7a17de4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:52.811183 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ae4696042de446dbb711b9ce3ee33e3c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.812770 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae4696042de446dbb711b9ce3ee33e3c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:52.817062 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_3edd4511aea94058a799c4a3de95f968"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.818533 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3edd4511aea94058a799c4a3de95f968 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:52.823721 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 3423379d1e3d4020a4241bf8917f1050 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3423379d1e3d4020a4241bf8917f1050 already in progress: creating tablet
W20260430 19:58:52.825166 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 50d8b9a982fe4810858d1ec2b28d7894 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50d8b9a982fe4810858d1ec2b28d7894 already in progress: creating tablet
I20260430 19:58:52.833809 4818 tablet_bootstrap.cc:654] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.838723 5098 raft_consensus.cc:493] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.839161 5098 raft_consensus.cc:515] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.841444 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:52.842849 4158 leader_election.cc:343] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.846364 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:52.847899 4161 leader_election.cc:343] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.848341 4161 leader_election.cc:304] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.849196 5101 raft_consensus.cc:2749] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.849836 5098 leader_election.cc:290] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:52.855373 4729 tablet_bootstrap.cc:654] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:52.860651 4818 tablet_bootstrap.cc:492] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:52.861135 4818 ts_tablet_manager.cc:1403] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.088s user 0.009s sys 0.000s
I20260430 19:58:52.865618 4818 raft_consensus.cc:359] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.866263 4818 raft_consensus.cc:385] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:52.866676 4818 raft_consensus.cc:740] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:52.873309 4223 raft_consensus.cc:3060] T 654df09f602b4f96bcf5d13a514df55c P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:52.887234 4818 consensus_queue.cc:260] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.895553 4729 tablet_bootstrap.cc:492] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:52.896565 5092 raft_consensus.cc:493] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.896987 5092 raft_consensus.cc:515] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.898753 5092 leader_election.cc:290] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:52.900292 4024 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
W20260430 19:58:52.901278 4317 leader_election.cc:343] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 6220177b3e3e4d18bd8e20e6d9d96be7
I20260430 19:58:52.902276 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 186f9a32199e48a7933cc1c265dbdc2c (DEFAULT_TABLE table=default.Table_6fbda673636e42419e82c47142a45243 [id=aa0e5b5c7a0942cba126f34a3626da80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.902619 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6220177b3e3e4d18bd8e20e6d9d96be7" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:52.903215 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186f9a32199e48a7933cc1c265dbdc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.903156 4222 raft_consensus.cc:2468] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:52.903406 4024 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 0832f9191b7f4c3683f37cce8966b266 (table default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]). Expected version 1 got 0
I20260430 19:58:52.904166 4320 leader_election.cc:304] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:52.904107 4024 catalog_manager.cc:5671] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:52.905031 5092 raft_consensus.cc:2804] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:52.905310 5092 raft_consensus.cc:493] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:52.905580 5092 raft_consensus.cc:3060] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:52.895928 5053 raft_consensus.cc:493] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.910979 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 186f9a32199e48a7933cc1c265dbdc2c (DEFAULT_TABLE table=default.Table_6fbda673636e42419e82c47142a45243 [id=aa0e5b5c7a0942cba126f34a3626da80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.910849 5053 raft_consensus.cc:515] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.911932 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186f9a32199e48a7933cc1c265dbdc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.912546 5053 leader_election.cc:290] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:52.914633 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e87fda401047cfa7070ae95b1701a9" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:52.916887 4470 leader_election.cc:343] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.901881 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 186f9a32199e48a7933cc1c265dbdc2c (DEFAULT_TABLE table=default.Table_6fbda673636e42419e82c47142a45243 [id=aa0e5b5c7a0942cba126f34a3626da80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.917610 4352 tablet_service.cc:1511] Processing CreateTablet for tablet ce8fc9213b1544cf97e1a15cf38617c3 (DEFAULT_TABLE table=default.Table_dfb4852e9eaa4eb482610e15c43c2e87 [id=ee33a7b0a4d445319333e442da849d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.918084 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186f9a32199e48a7933cc1c265dbdc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.918560 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8fc9213b1544cf97e1a15cf38617c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.920548 4818 ts_tablet_manager.cc:1434] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.059s user 0.007s sys 0.000s
I20260430 19:58:52.910120 4514 tablet_service.cc:1511] Processing CreateTablet for tablet ce8fc9213b1544cf97e1a15cf38617c3 (DEFAULT_TABLE table=default.Table_dfb4852e9eaa4eb482610e15c43c2e87 [id=ee33a7b0a4d445319333e442da849d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:52.923436 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8fc9213b1544cf97e1a15cf38617c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:52.923645 4026 master_service.cc:954] ReplaceTablet: received request to replace tablet fa59394fc34f419ca22a695b8dff4613 from {username='slave'} at 127.0.0.1:55130
W20260430 19:58:52.927991 4470 leader_election.cc:343] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: e4e87fda401047cfa7070ae95b1701a9
I20260430 19:58:52.928442 4470 leader_election.cc:304] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.929786 5053 raft_consensus.cc:2749] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.932425 4027 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 6f39fb64a301447f988c53ccfe896074
I20260430 19:58:52.934772 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f39fb64a301447f988c53ccfe896074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:52.936221 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6f39fb64a301447f988c53ccfe896074 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f39fb64a301447f988c53ccfe896074 already in progress: creating tablet
I20260430 19:58:52.938911 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f39fb64a301447f988c53ccfe896074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:52.941026 4020 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
I20260430 19:58:52.941841 4020 catalog_manager.cc:5671] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2). New cstate: current_term: 1 leader_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:52.943466 4193 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f39fb64a301447f988c53ccfe896074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:52.945283 4020 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:52.946163 4020 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 0832f9191b7f4c3683f37cce8966b266 (table default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]). Expected version 1 got 0
W20260430 19:58:52.947157 4021 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 239a2c6a534f4f478e436d14f94cd762 consensus state has no leader
I20260430 19:58:52.948139 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet a4b0d830f1b14ce382b2e0242f6565e7
I20260430 19:58:52.950446 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet a4b0d830f1b14ce382b2e0242f6565e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:52.946261 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 6f39fb64a301447f988c53ccfe896074 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f39fb64a301447f988c53ccfe896074 already in progress: creating tablet
I20260430 19:58:52.951227 4915 tablet_replica.cc:333] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:52.951905 4915 raft_consensus.cc:2243] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:52.952169 4193 tablet_service.cc:1558] Processing DeleteTablet for tablet a4b0d830f1b14ce382b2e0242f6565e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:52.952401 4915 raft_consensus.cc:2272] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:52.952823 4742 tablet_replica.cc:333] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:52.953531 4742 raft_consensus.cc:2243] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:52.953976 4742 raft_consensus.cc:2272] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:52.954107 4915 ts_tablet_manager.cc:1916] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:52.955904 4742 ts_tablet_manager.cc:1916] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:52.938081 5092 raft_consensus.cc:515] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.963080 4960 raft_consensus.cc:493] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:52.964357 4960 raft_consensus.cc:515] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:52.966539 4354 tablet_service.cc:1558] Processing DeleteTablet for tablet a4b0d830f1b14ce382b2e0242f6565e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:52.967839 4729 ts_tablet_manager.cc:1403] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.208s user 0.010s sys 0.000s
I20260430 19:58:52.969802 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:52.970001 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:52.971235 4161 leader_election.cc:343] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:52.971696 4158 leader_election.cc:343] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:52.972026 4158 leader_election.cc:304] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:52.972683 5098 raft_consensus.cc:2749] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:52.975262 4960 leader_election.cc:290] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:52.979692 4818 tablet_bootstrap.cc:492] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:52.990275 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_0c0f3cc776e847188b388e9ba7cb1285"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:52.991874 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c0f3cc776e847188b388e9ba7cb1285 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:52.994899 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 6f39fb64a301447f988c53ccfe896074 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f39fb64a301447f988c53ccfe896074 already in progress: creating tablet
W20260430 19:58:52.997502 4317 leader_election.cc:343] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 6220177b3e3e4d18bd8e20e6d9d96be7
I20260430 19:58:52.999398 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6220177b3e3e4d18bd8e20e6d9d96be7" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:53.000036 4223 raft_consensus.cc:3060] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:53.012627 4776 tablet_bootstrap.cc:492] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:53.013047 4776 ts_tablet_manager.cc:1403] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.266s user 0.019s sys 0.049s
I20260430 19:58:53.013049 4818 tablet_bootstrap.cc:654] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.014250 5092 leader_election.cc:290] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:53.015236 4776 raft_consensus.cc:359] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.015834 4776 raft_consensus.cc:385] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.016067 4776 raft_consensus.cc:740] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.016623 4776 consensus_queue.cc:260] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.018412 4776 ts_tablet_manager.cc:1434] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260430 19:58:53.020191 4776 tablet_bootstrap.cc:492] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:53.027060 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet fa59394fc34f419ca22a695b8dff4613
I20260430 19:58:53.028242 4026 catalog_manager.cc:6741] ReplaceTablet: tablet fa59394fc34f419ca22a695b8dff4613 deleted and replaced by tablet d846bbe4af0544e88ea5012ccc815881
I20260430 19:58:53.030246 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet fa59394fc34f419ca22a695b8dff4613 with delete_type TABLET_DATA_DELETED (replaced by tablet d846bbe4af0544e88ea5012ccc815881) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.033471 4029 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 0832f9191b7f4c3683f37cce8966b266 (table default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]). Expected version 1 got 0
I20260430 19:58:53.033865 4776 tablet_bootstrap.cc:654] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.035897 4029 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:53.039703 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet fa59394fc34f419ca22a695b8dff4613 with delete_type TABLET_DATA_DELETED (replaced by tablet d846bbe4af0544e88ea5012ccc815881) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.039961 4029 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
I20260430 19:58:53.041074 4915 ts_tablet_manager.cc:1929] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:53.041417 4915 log.cc:1199] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/a4b0d830f1b14ce382b2e0242f6565e7
I20260430 19:58:53.042073 4915 ts_tablet_manager.cc:1950] T a4b0d830f1b14ce382b2e0242f6565e7 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:53.044895 4024 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074] alter complete (version 2)
I20260430 19:58:53.050582 4729 raft_consensus.cc:359] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.053555 4729 raft_consensus.cc:385] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.053835 4729 raft_consensus.cc:740] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.054452 4729 consensus_queue.cc:260] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.055393 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet fa59394fc34f419ca22a695b8dff4613 with delete_type TABLET_DATA_DELETED (replaced by tablet d846bbe4af0544e88ea5012ccc815881) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.058991 4818 tablet_bootstrap.cc:492] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:53.063930 5117 tablet_replica.cc:333] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:53.064774 5117 raft_consensus.cc:2243] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:53.065253 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_8a76e0b1a42d4a27b12f435c0fef102b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:53.066761 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a76e0b1a42d4a27b12f435c0fef102b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.071094 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_6ca128f6264545baa6d5b0dfe8023a8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:53.072548 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ca128f6264545baa6d5b0dfe8023a8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.080513 4742 ts_tablet_manager.cc:1929] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:53.081702 4742 log.cc:1199] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/a4b0d830f1b14ce382b2e0242f6565e7
I20260430 19:58:53.082983 4742 ts_tablet_manager.cc:1950] T a4b0d830f1b14ce382b2e0242f6565e7 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:53.072896 4818 ts_tablet_manager.cc:1403] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.095s user 0.009s sys 0.002s
I20260430 19:58:53.094678 5117 raft_consensus.cc:2272] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:53.096257 4818 raft_consensus.cc:359] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.096858 5117 ts_tablet_manager.cc:1916] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:53.096868 4818 raft_consensus.cc:385] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.098001 4818 raft_consensus.cc:740] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.098694 4818 consensus_queue.cc:260] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.100216 4358 tablet_service.cc:1511] Processing CreateTablet for tablet f3df3bbfb42d4a8fae61b6dd1cd2c776 (DEFAULT_TABLE table=default.Table_e8541719669c46e2bcd98610f2c4181b [id=17467d8bd04945169c6587ac209dff36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.102850 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_5ff207a6e7ed439692d024dfc07accae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:53.103941 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3df3bbfb42d4a8fae61b6dd1cd2c776. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:53.104504 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ff207a6e7ed439692d024dfc07accae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.117892 4915 tablet_replica.cc:333] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:53.118574 4915 raft_consensus.cc:2243] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:53.119421 4915 raft_consensus.cc:2272] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:53.121255 4024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.3:35079 (request call id 37) took 805 ms (client timeout 1000 ms). Trace:
I20260430 19:58:53.121573 4915 ts_tablet_manager.cc:1916] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:53.123256 4729 ts_tablet_manager.cc:1434] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.155s user 0.006s sys 0.000s
I20260430 19:58:53.124089 4729 tablet_bootstrap.cc:492] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:53.124276 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet a4b0d830f1b14ce382b2e0242f6565e7 (table default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]) successfully deleted
I20260430 19:58:53.125800 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet c81b11280f6d49e681537c88fb1cbf68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.126319 4020 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_cf25ab133dde4d9aa99dabc520b379aa [id=3a1eab4e333c4999b8b5155702ad0e3b] alter complete (version 1)
I20260430 19:58:53.126962 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
W20260430 19:58:53.128083 4020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.2:49905 (request call id 41) took 959 ms (client timeout 1000 ms). Trace:
I20260430 19:58:53.128158 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c119dd9a5d944eabd588f62a76da98e with delete_type TABLET_DATA_DELETED (replaced by tablet 239a2c6a534f4f478e436d14f94cd762) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.129796 4818 ts_tablet_manager.cc:1434] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.036s user 0.005s sys 0.000s
I20260430 19:58:53.130142 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee21f80531c49b7b68f0fc7e5cdeabf (DEFAULT_TABLE table=default.Table_fe61a348f64e4406ab80425071253c23 [id=4b2819ce188948c9b98c8df597c6f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.131354 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee21f80531c49b7b68f0fc7e5cdeabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.137262 4818 tablet_bootstrap.cc:492] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:53.138357 4021 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4f04fa211ec3447488b05b4f5b157487" } modify_external_catalogs: true
I20260430 19:58:53.138875 4021 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4f04fa211ec3447488b05b4f5b157487" } modify_external_catalogs: true
I20260430 19:58:53.142339 4818 tablet_bootstrap.cc:654] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.152299 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_c8f20997cfad46fea29e3a23559df569" } new_table_name: "default.Table_7937044cd31c4c619c6dfc1a8411647d" modify_external_catalogs: true
I20260430 19:58:53.155683 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_830b99b7bf1e49e8ad90881ae777dca7" } modify_external_catalogs: true
I20260430 19:58:53.156198 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_830b99b7bf1e49e8ad90881ae777dca7" } modify_external_catalogs: true
I20260430 19:58:53.134289 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.170017 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee21f80531c49b7b68f0fc7e5cdeabf (DEFAULT_TABLE table=default.Table_fe61a348f64e4406ab80425071253c23 [id=4b2819ce188948c9b98c8df597c6f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.171250 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee21f80531c49b7b68f0fc7e5cdeabf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.175339 4514 tablet_service.cc:1511] Processing CreateTablet for tablet f3df3bbfb42d4a8fae61b6dd1cd2c776 (DEFAULT_TABLE table=default.Table_e8541719669c46e2bcd98610f2c4181b [id=17467d8bd04945169c6587ac209dff36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.176503 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3df3bbfb42d4a8fae61b6dd1cd2c776. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:53.134927 4020 rpcz_store.cc:269] 0430 19:58:52.168269 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:52.258582 (+ 90313us) service_pool.cc:225] Handling call
0430 19:58:52.346029 (+ 87447us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:53.128024 (+781995us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:52.954293 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:52.954695 (+ 402us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:52.954716 (+ 21us) write_op.cc:435] Acquired schema lock
0430 19:58:52.954727 (+ 11us) tablet.cc:662] Decoding operations
0430 19:58:52.954961 (+ 234us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:52.954995 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:52.955017 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:52.955145 (+ 128us) tablet.cc:701] Row locks acquired
0430 19:58:52.955158 (+ 13us) write_op.cc:260] PREPARE: finished
0430 19:58:52.955256 (+ 98us) write_op.cc:270] Start()
0430 19:58:52.955328 (+ 72us) write_op.cc:276] Timestamp: P: 1777579132955173 usec, L: 0
0430 19:58:52.955343 (+ 15us) op_driver.cc:348] REPLICATION: starting
0430 19:58:52.955935 (+ 592us) log.cc:844] Serialized 527 byte log entry
0430 19:58:53.016454 (+ 60519us) op_driver.cc:464] REPLICATION: finished
0430 19:58:53.018647 (+ 2193us) write_op.cc:301] APPLY: starting
0430 19:58:53.018693 (+ 46us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:53.018774 (+ 81us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:53.018846 (+ 72us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:53.018993 (+ 147us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:53.019104 (+ 111us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:53.019142 (+ 38us) write_op.cc:312] APPLY: finished
0430 19:58:53.019294 (+ 152us) log.cc:844] Serialized 23 byte log entry
0430 19:58:53.019411 (+ 117us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:53.019462 (+ 51us) write_op.cc:454] Released schema lock
0430 19:58:53.019488 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:53.019694 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:53.019720 (+ 26us) write_op.cc:454] Released schema lock
Related trace 'op':
0430 19:58:53.034078 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:53.034447 (+ 369us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:53.034470 (+ 23us) write_op.cc:435] Acquired schema lock
0430 19:58:53.034481 (+ 11us) tablet.cc:662] Decoding operations
0430 19:58:53.034775 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:53.034814 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:53.034837 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:53.034994 (+ 157us) tablet.cc:701] Row locks acquired
0430 19:58:53.035006 (+ 12us) write_op.cc:260] PREPARE: finished
0430 19:58:53.035121 (+ 115us) write_op.cc:270] Start()
0430 19:58:53.035205 (+ 84us) write_op.cc:276] Timestamp: P: 1777579133035024 usec, L: 0
0430 19:58:53.035221 (+ 16us) op_driver.cc:348] REPLICATION: starting
0430 19:58:53.046869 (+ 11648us) log.cc:844] Serialized 356 byte log entry
0430 19:58:53.048177 (+ 1308us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927268951 00007f29272fb208 00007f292776d9f1 00007f292776d1c8 00007f2927771e61 00007f2927771e19 00007f2927771da9 00007f2927771d71 00007f2927770f9d 00000000005b3bc4 00000000005b39f9
0430 19:58:53.087452 (+ 39275us) op_driver.cc:464] REPLICATION: finished
0430 19:58:53.107637 (+ 20185us) write_op.cc:301] APPLY: starting
0430 19:58:53.107694 (+ 57us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:53.107782 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:53.107802 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:53.107964 (+ 162us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:53.108082 (+ 118us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:53.108124 (+ 42us) write_op.cc:312] APPLY: finished
0430 19:58:53.108292 (+ 168us) log.cc:844] Serialized 23 byte log entry
0430 19:58:53.115279 (+ 6987us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:53.115344 (+ 65us) write_op.cc:454] Released schema lock
0430 19:58:53.115375 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:53.115592 (+ 217us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:53.115619 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":197077888,"child_traces":[["op",{"apply.queue_time_us":2041,"apply.run_cpu_time_us":1056,"apply.run_wall_time_us":1056,"num_ops":1,"prepare.queue_time_us":2383,"prepare.run_cpu_time_us":1830,"prepare.run_wall_time_us":1831,"replication_time_us":61039}],["op",{"apply.queue_time_us":20021,"apply.run_cpu_time_us":1142,"apply.run_wall_time_us":7927,"num_ops":1,"prepare.queue_time_us":10267,"prepare.run_cpu_time_us":3474,"prepare.run_wall_time_us":14216,"replication_time_us":52138,"spinlock_wait_cycles":25642880}]]}
W20260430 19:58:53.121670 4024 rpcz_store.cc:269] 0430 19:58:52.316262 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:52.316365 (+ 103us) service_pool.cc:225] Handling call
0430 19:58:52.343708 (+ 27343us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:52.910786 (+567078us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b5400030300. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f29276a1671 00007f2927775ff9 00007f292776bc1d 00007f292772d27b 00007f292772d054 00007f292914f7d1 00007f292915002f 00007f292901326b 00007f2929116cae 00007f2926d1d4a4 00007f2926d1d432
0430 19:58:53.121195 (+210409us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:52.911064 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:52.911386 (+ 322us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:52.911406 (+ 20us) write_op.cc:435] Acquired schema lock
0430 19:58:52.911418 (+ 12us) tablet.cc:662] Decoding operations
0430 19:58:52.911716 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:52.911771 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:52.911791 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:52.911943 (+ 152us) tablet.cc:701] Row locks acquired
0430 19:58:52.911956 (+ 13us) write_op.cc:260] PREPARE: finished
0430 19:58:52.912052 (+ 96us) write_op.cc:270] Start()
0430 19:58:52.912122 (+ 70us) write_op.cc:276] Timestamp: P: 1777579132911978 usec, L: 0
0430 19:58:52.912136 (+ 14us) op_driver.cc:348] REPLICATION: starting
0430 19:58:52.912536 (+ 400us) log.cc:844] Serialized 527 byte log entry
0430 19:58:52.930515 (+ 17979us) op_driver.cc:464] REPLICATION: finished
0430 19:58:52.934751 (+ 4236us) write_op.cc:301] APPLY: starting
0430 19:58:52.934802 (+ 51us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:52.934914 (+ 112us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:52.934991 (+ 77us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:52.935147 (+ 156us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:52.935271 (+ 124us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:52.935312 (+ 41us) write_op.cc:312] APPLY: finished
0430 19:58:52.935471 (+ 159us) log.cc:844] Serialized 23 byte log entry
0430 19:58:52.935590 (+ 119us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:52.935646 (+ 56us) write_op.cc:454] Released schema lock
0430 19:58:52.935680 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:52.935890 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:52.935913 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0430 19:58:52.943182 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:52.943569 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:52.943607 (+ 38us) write_op.cc:435] Acquired schema lock
0430 19:58:52.943620 (+ 13us) tablet.cc:662] Decoding operations
0430 19:58:52.943896 (+ 276us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:52.943935 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:52.943958 (+ 23us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:52.944095 (+ 137us) tablet.cc:701] Row locks acquired
0430 19:58:52.944107 (+ 12us) write_op.cc:260] PREPARE: finished
0430 19:58:52.944210 (+ 103us) write_op.cc:270] Start()
0430 19:58:52.944283 (+ 73us) write_op.cc:276] Timestamp: P: 1777579132944130 usec, L: 0
0430 19:58:52.944298 (+ 15us) op_driver.cc:348] REPLICATION: starting
0430 19:58:52.944764 (+ 466us) log.cc:844] Serialized 356 byte log entry
0430 19:58:53.016236 (+ 71472us) op_driver.cc:464] REPLICATION: finished
0430 19:58:53.017181 (+ 945us) write_op.cc:301] APPLY: starting
0430 19:58:53.017241 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:53.017326 (+ 85us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:53.017401 (+ 75us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:53.017566 (+ 165us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:53.017700 (+ 134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:53.017736 (+ 36us) write_op.cc:312] APPLY: finished
0430 19:58:53.017919 (+ 183us) log.cc:844] Serialized 23 byte log entry
0430 19:58:53.018069 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:53.018123 (+ 54us) write_op.cc:454] Released schema lock
0430 19:58:53.018155 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:53.018360 (+ 205us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:53.018386 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":63730688,"child_traces":[["op",{"apply.queue_time_us":4080,"apply.run_cpu_time_us":1114,"apply.run_wall_time_us":1115,"num_ops":1,"prepare.queue_time_us":96,"prepare.run_cpu_time_us":1659,"prepare.run_wall_time_us":1658,"replication_time_us":18309}],["op",{"apply.queue_time_us":785,"apply.run_cpu_time_us":1151,"apply.run_wall_time_us":1151,"num_ops":1,"prepare.queue_time_us":4861,"prepare.run_cpu_time_us":1789,"prepare.run_wall_time_us":1789,"replication_time_us":71845}]]}
I20260430 19:58:53.208453 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 04cf61c355a34941b930167ea65624fa (DEFAULT_TABLE table=default.Table_f025a41a9b8044ba9e39f5634daa2a55 [id=9188fefc37c64138ad871d4f609a9504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.209801 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04cf61c355a34941b930167ea65624fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.209867 4818 tablet_bootstrap.cc:492] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:53.210268 4818 ts_tablet_manager.cc:1403] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.073s user 0.008s sys 0.000s
I20260430 19:58:53.211683 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 04cf61c355a34941b930167ea65624fa (DEFAULT_TABLE table=default.Table_f025a41a9b8044ba9e39f5634daa2a55 [id=9188fefc37c64138ad871d4f609a9504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.213006 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04cf61c355a34941b930167ea65624fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.216701 4915 ts_tablet_manager.cc:1929] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260430 19:58:53.217103 4915 log.cc:1199] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/fa59394fc34f419ca22a695b8dff4613
I20260430 19:58:53.217748 4915 ts_tablet_manager.cc:1950] T fa59394fc34f419ca22a695b8dff4613 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:53.219089 5092 raft_consensus.cc:493] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:53.219090 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074]) successfully deleted
I20260430 19:58:53.219506 5092 raft_consensus.cc:515] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.221499 5092 leader_election.cc:290] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:53.223470 4999 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 3 FOLLOWER]: Advancing to term 4
W20260430 19:58:53.224416 4320 leader_election.cc:343] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: 37d6c54ae89946158cad45e741c1c0f0
W20260430 19:58:53.225358 4317 leader_election.cc:343] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 37d6c54ae89946158cad45e741c1c0f0
I20260430 19:58:53.225716 4317 leader_election.cc:304] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:53.226780 5092 raft_consensus.cc:2749] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:53.093494 4223 raft_consensus.cc:2468] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:53.102070 4193 tablet_service.cc:1511] Processing CreateTablet for tablet f3df3bbfb42d4a8fae61b6dd1cd2c776 (DEFAULT_TABLE table=default.Table_e8541719669c46e2bcd98610f2c4181b [id=17467d8bd04945169c6587ac209dff36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.231501 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3df3bbfb42d4a8fae61b6dd1cd2c776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.237149 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 0832f9191b7f4c3683f37cce8966b266 (table default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16]). Expected version 1 got 0
I20260430 19:58:53.239437 4320 leader_election.cc:304] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:53.240458 5092 raft_consensus.cc:2804] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:53.240909 5092 raft_consensus.cc:697] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:53.241637 5092 consensus_queue.cc:237] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.242113 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:53.243911 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
W20260430 19:58:53.249171 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet c81b11280f6d49e681537c88fb1cbf68 because the tablet was not found. No further retry: Not found: Tablet not found: c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:53.249325 4020 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_db210d74dff541908dd0daed2a35c325" } new_table_name: "default.Table_cf9ef037d1634978a0ddbe0a06f32284" modify_external_catalogs: true
I20260430 19:58:53.104080 4203 tablet_service.cc:1511] Processing CreateTablet for tablet ce8fc9213b1544cf97e1a15cf38617c3 (DEFAULT_TABLE table=default.Table_dfb4852e9eaa4eb482610e15c43c2e87 [id=ee33a7b0a4d445319333e442da849d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.171310 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee21f80531c49b7b68f0fc7e5cdeabf (DEFAULT_TABLE table=default.Table_fe61a348f64e4406ab80425071253c23 [id=4b2819ce188948c9b98c8df597c6f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.255383 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee21f80531c49b7b68f0fc7e5cdeabf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:53.257618 4029 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.1:59387 (request call id 48) took 967 ms (client timeout 1000 ms). Trace:
I20260430 19:58:53.229961 4190 tablet_service.cc:1511] Processing CreateTablet for tablet 04cf61c355a34941b930167ea65624fa (DEFAULT_TABLE table=default.Table_f025a41a9b8044ba9e39f5634daa2a55 [id=9188fefc37c64138ad871d4f609a9504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.260013 4776 tablet_bootstrap.cc:492] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
W20260430 19:58:53.258765 4029 rpcz_store.cc:269] 0430 19:58:52.290741 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:52.295556 (+ 4815us) service_pool.cc:225] Handling call
0430 19:58:52.351725 (+ 56169us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:53.257550 (+905825us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":14,"spinlock_wait_cycles":123104640}
I20260430 19:58:53.261564 4776 ts_tablet_manager.cc:1403] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.242s user 0.020s sys 0.016s
W20260430 19:58:53.265594 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 09f272340cd942e9b2af9e38a2bdec82 because the tablet was not found. No further retry: Not found: Tablet not found: 09f272340cd942e9b2af9e38a2bdec82
I20260430 19:58:53.268200 4776 raft_consensus.cc:359] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:53.268848 4776 raft_consensus.cc:385] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.260426 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04cf61c355a34941b930167ea65624fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.270035 4192 tablet_service.cc:1558] Processing DeleteTablet for tablet c81b11280f6d49e681537c88fb1cbf68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.271476 4192 tablet_service.cc:1558] Processing DeleteTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.271476 4024 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:53.275837 4192 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.278731 4192 tablet_service.cc:1558] Processing DeleteTablet for tablet 404f20adf9d741f3a036b06cbdbaedb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.279280 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet e4e87fda401047cfa7070ae95b1701a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.279956 4915 tablet_replica.cc:333] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:53.280115 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8fc9213b1544cf97e1a15cf38617c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.280289 4192 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 with delete_type TABLET_DATA_DELETED (replaced by tablet cfd78a1cd7a94e27bb376d1b7c151073) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.280633 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a73bf5f11b0e46198f70577bc06dcf91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:53.282243 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a73bf5f11b0e46198f70577bc06dcf91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.282667 4742 tablet_replica.cc:333] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:53.269109 4776 raft_consensus.cc:740] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.290957 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 with delete_type TABLET_DATA_DELETED (replaced by tablet cfd78a1cd7a94e27bb376d1b7c151073) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.292797 4776 consensus_queue.cc:260] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:53.304143 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet c81b11280f6d49e681537c88fb1cbf68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.304807 4776 ts_tablet_manager.cc:1434] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.043s user 0.005s sys 0.000s
I20260430 19:58:53.304809 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet a4b0d830f1b14ce382b2e0242f6565e7 (table default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]) successfully deleted
I20260430 19:58:53.305629 4776 tablet_bootstrap.cc:492] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:53.311345 4742 raft_consensus.cc:2243] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:53.311879 4742 raft_consensus.cc:2272] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:53.322427 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 37d6c54ae89946158cad45e741c1c0f0 with delete_type TABLET_DATA_DELETED (replaced by tablet bdf98515d8284e3b9b84480807e8108a) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.323104 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 6220177b3e3e4d18bd8e20e6d9d96be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.213020 4818 raft_consensus.cc:359] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.323840 4818 raft_consensus.cc:385] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.323732 4352 tablet_service.cc:1558] Processing DeleteTablet for tablet f3db0439f4f042a4a9cb38ebd3764ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.324101 4818 raft_consensus.cc:740] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
W20260430 19:58:53.328845 4470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 131231us
I20260430 19:58:53.338066 4776 tablet_bootstrap.cc:654] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.347141 4742 ts_tablet_manager.cc:1916] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:53.347441 4503 tablet_service.cc:1511] Processing CreateTablet for tablet 094062a2d0f048939b5202e3154bbfd8 (DEFAULT_TABLE table=default.Table_a3dc085505d24384b373f9edc7361950 [id=aa62c256a9c14d01bcb9ba37df7a340f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.348534 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094062a2d0f048939b5202e3154bbfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.351049 4915 raft_consensus.cc:2243] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:53.351481 4915 raft_consensus.cc:2272] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:53.352281 4024 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:53.353034 4915 ts_tablet_manager.cc:1916] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:53.355697 4024 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
I20260430 19:58:53.357129 4352 tablet_service.cc:1511] Processing CreateTablet for tablet 094062a2d0f048939b5202e3154bbfd8 (DEFAULT_TABLE table=default.Table_a3dc085505d24384b373f9edc7361950 [id=aa62c256a9c14d01bcb9ba37df7a340f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.358196 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094062a2d0f048939b5202e3154bbfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.356770 4024 catalog_manager.cc:5671] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a7356e99e744525861e12c108cfa3fe (127.3.190.1). New cstate: current_term: 1 leader_uuid: "5a7356e99e744525861e12c108cfa3fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: HEALTHY } } }
I20260430 19:58:53.361198 4024 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet db0efaecfef44457a5128e878796c061 (table default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c [id=d4bfa57fdfc64108ba1e909d0230fb51]). Expected version 4 got 0
I20260430 19:58:53.367205 4729 tablet_bootstrap.cc:654] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.381932 5136 tablet.cc:1724] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260430 19:58:53.327253 4818 consensus_queue.cc:260] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.386580 4818 ts_tablet_manager.cc:1434] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.176s user 0.007s sys 0.000s
I20260430 19:58:53.388320 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 094062a2d0f048939b5202e3154bbfd8 (DEFAULT_TABLE table=default.Table_a3dc085505d24384b373f9edc7361950 [id=aa62c256a9c14d01bcb9ba37df7a340f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.400605 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094062a2d0f048939b5202e3154bbfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.407963 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet 443308f2a26f4622aa4551ff5f73fc39 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:53.416198 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet db0efaecfef44457a5128e878796c061 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:53.422246 4021 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 51e35e0981e649f684caa96499040c15
I20260430 19:58:53.424890 4818 tablet_bootstrap.cc:492] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:53.425292 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 51e35e0981e649f684caa96499040c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.426862 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet f50116c63b0747099de2d234ca3faaac
I20260430 19:58:53.431012 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet f50116c63b0747099de2d234ca3faaac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:53.431535 4027 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4758b3d846434b2b8466033c083b7339 consensus state has no leader
I20260430 19:58:53.437727 4915 ts_tablet_manager.cc:1929] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:53.438145 4915 log.cc:1199] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/e4e87fda401047cfa7070ae95b1701a9
W20260430 19:58:53.444365 4470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 114410us
I20260430 19:58:53.448807 4436 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:53.430459 4818 tablet_bootstrap.cc:654] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.460605 5092 raft_consensus.cc:493] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:53.461061 5092 raft_consensus.cc:515] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.462872 5092 leader_election.cc:290] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:53.465013 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafc058f392a45de9d30b521ebde0950" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:53.465615 4530 raft_consensus.cc:2468] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:53.467036 4317 leader_election.cc:304] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:53.467862 5092 raft_consensus.cc:2804] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:53.468164 5092 raft_consensus.cc:493] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:53.468443 5092 raft_consensus.cc:3060] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:53.474942 4915 ts_tablet_manager.cc:1950] T e4e87fda401047cfa7070ae95b1701a9 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:53.478927 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet e4e87fda401047cfa7070ae95b1701a9 (table default.Table_4b6fc137cf6945079fa8310db94a2b1a [id=2cb52e96af9f4eea998718eef0ac58ad]) successfully deleted
I20260430 19:58:53.480111 4915 tablet_replica.cc:333] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:53.480891 4915 raft_consensus.cc:2243] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:53.477407 5092 raft_consensus.cc:515] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:53.488624 4016 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.61:35559, user_credentials={real_user=slave}} blocked reactor thread for 73303.4us
I20260430 19:58:53.490284 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 51e35e0981e649f684caa96499040c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.491394 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 51e35e0981e649f684caa96499040c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.486723 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafc058f392a45de9d30b521ebde0950" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:53.492014 4530 raft_consensus.cc:3060] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:53.499819 4530 raft_consensus.cc:2468] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:53.505129 4915 raft_consensus.cc:2272] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:53.507187 4915 ts_tablet_manager.cc:1916] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:53.512681 4503 tablet_service.cc:1511] Processing CreateTablet for tablet e1607e4cb87b4b8f9b35a1aee6128cca (DEFAULT_TABLE table=default.Table_e36d0112106b45e99f2ec17696ab9aac [id=83ffa74d0e914258aaf41dd30f650535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.513705 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1607e4cb87b4b8f9b35a1aee6128cca. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:53.520761 4020 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ebd991541273429e9c2bdc018ee402f5 has no consensus state
I20260430 19:58:53.525013 5136 tablet.cc:1724] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:53.526291 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_50723dc3e29240dbba36ca94e7b72a3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:53.527753 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50723dc3e29240dbba36ca94e7b72a3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.530563 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_968941fa27b84dfab105dbb83df734ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:53.533564 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_968941fa27b84dfab105dbb83df734ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.541767 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet f50116c63b0747099de2d234ca3faaac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:53.543776 4729 tablet_bootstrap.cc:492] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:53.544170 4729 ts_tablet_manager.cc:1403] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.420s user 0.009s sys 0.000s
I20260430 19:58:53.502537 4317 leader_election.cc:304] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:53.546036 5014 raft_consensus.cc:2804] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:53.498921 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet f50116c63b0747099de2d234ca3faaac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.491606 5148 tablet.cc:1724] T c78d50690da7428fadb848dadefec47c P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260430 19:58:53.567821 4356 tablet_service.cc:1511] Processing CreateTablet for tablet e1607e4cb87b4b8f9b35a1aee6128cca (DEFAULT_TABLE table=default.Table_e36d0112106b45e99f2ec17696ab9aac [id=83ffa74d0e914258aaf41dd30f650535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.568846 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1607e4cb87b4b8f9b35a1aee6128cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.572258 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 84b02b29777341d3934f6a77967d79b0 (DEFAULT_TABLE table=default.Table_6e4f222e6ba54644a2e2da32762001f1 [id=67d7c591ac744a4c9d8249fad5a4709b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.573257 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84b02b29777341d3934f6a77967d79b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.577288 4354 tablet_service.cc:1511] Processing CreateTablet for tablet 3d36c2538ec84e0889c4e53975a2ff1c (DEFAULT_TABLE table=default.Table_90a4740b66574861a91bc789cb56b353 [id=e714321133124344bc43ba5900300ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.578341 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d36c2538ec84e0889c4e53975a2ff1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.592769 4203 tablet_service.cc:1511] Processing CreateTablet for tablet e1607e4cb87b4b8f9b35a1aee6128cca (DEFAULT_TABLE table=default.Table_e36d0112106b45e99f2ec17696ab9aac [id=83ffa74d0e914258aaf41dd30f650535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.594296 4205 tablet_service.cc:1511] Processing CreateTablet for tablet 84b02b29777341d3934f6a77967d79b0 (DEFAULT_TABLE table=default.Table_6e4f222e6ba54644a2e2da32762001f1 [id=67d7c591ac744a4c9d8249fad5a4709b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.598102 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84b02b29777341d3934f6a77967d79b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.599960 4776 tablet_bootstrap.cc:492] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:53.600346 4776 ts_tablet_manager.cc:1403] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.295s user 0.012s sys 0.029s
I20260430 19:58:53.595333 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafc058f392a45de9d30b521ebde0950" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:53.603066 4223 raft_consensus.cc:2468] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:53.606799 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1607e4cb87b4b8f9b35a1aee6128cca. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:53.606786 5149 log.cc:927] Time spent T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Append to log took a long time: real 0.096s user 0.001s sys 0.000s
I20260430 19:58:53.609915 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafc058f392a45de9d30b521ebde0950" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:53.610894 4960 raft_consensus.cc:493] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:53.610467 4223 raft_consensus.cc:3060] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:53.611378 4960 raft_consensus.cc:515] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.613282 4960 leader_election.cc:290] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:53.613159 4776 raft_consensus.cc:359] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.613799 4776 raft_consensus.cc:385] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.614126 4776 raft_consensus.cc:740] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.614903 5117 ts_tablet_manager.cc:1929] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:53.615274 5117 log.cc:1199] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/a4b0d830f1b14ce382b2e0242f6565e7
I20260430 19:58:53.615458 4776 consensus_queue.cc:260] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.616027 5117 ts_tablet_manager.cc:1950] T a4b0d830f1b14ce382b2e0242f6565e7 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:53.616390 4190 tablet_service.cc:1511] Processing CreateTablet for tablet 3d36c2538ec84e0889c4e53975a2ff1c (DEFAULT_TABLE table=default.Table_90a4740b66574861a91bc789cb56b353 [id=e714321133124344bc43ba5900300ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.616866 4776 ts_tablet_manager.cc:1434] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260430 19:58:53.617565 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d36c2538ec84e0889c4e53975a2ff1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.628068 5092 leader_election.cc:290] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:53.628590 5092 raft_consensus.cc:493] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:53.628842 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_1ef1c0b4ac9f4d5ca9433f755ecf83fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:53.628978 5092 raft_consensus.cc:515] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:53.630347 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ef1c0b4ac9f4d5ca9433f755ecf83fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.633962 4021 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_7937044cd31c4c619c6dfc1a8411647d" } modify_external_catalogs: true
I20260430 19:58:53.634446 4021 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_7937044cd31c4c619c6dfc1a8411647d" } modify_external_catalogs: true
I20260430 19:58:53.641786 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5a90ad7e91c44b4ea6f4240ebe7e7d65" } modify_external_catalogs: true
I20260430 19:58:53.642262 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5a90ad7e91c44b4ea6f4240ebe7e7d65" } modify_external_catalogs: true
I20260430 19:58:53.642968 4026 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_2a59def4cb444084bf2b70663d1430b4" } new_table_name: "default.Table_36c4be64f98844e591eac49478ce90d5" modify_external_catalogs: true
I20260430 19:58:53.618332 4223 raft_consensus.cc:2468] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:53.619251 4776 tablet_bootstrap.cc:492] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:53.649324 5014 raft_consensus.cc:697] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:53.650097 4776 tablet_bootstrap.cc:654] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.650398 5014 consensus_queue.cc:237] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.654765 4960 raft_consensus.cc:493] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:53.655292 4960 raft_consensus.cc:515] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.657501 4960 leader_election.cc:290] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:53.659189 5117 tablet_replica.cc:333] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:53.644091 4742 ts_tablet_manager.cc:1929] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260430 19:58:53.661263 4742 log.cc:1199] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/fa59394fc34f419ca22a695b8dff4613
I20260430 19:58:53.662156 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745f04d04091443ca1c39949d005dd69" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:53.662936 4532 raft_consensus.cc:2393] T 745f04d04091443ca1c39949d005dd69 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8345ec7f57c4399a8a4afe4dd43439e in current term 1: Already voted for candidate 5a7356e99e744525861e12c108cfa3fe in this term.
I20260430 19:58:53.663205 4742 ts_tablet_manager.cc:1950] T fa59394fc34f419ca22a695b8dff4613 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:53.662977 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 3d36c2538ec84e0889c4e53975a2ff1c (DEFAULT_TABLE table=default.Table_90a4740b66574861a91bc789cb56b353 [id=e714321133124344bc43ba5900300ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.664407 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 84b02b29777341d3934f6a77967d79b0 (DEFAULT_TABLE table=default.Table_6e4f222e6ba54644a2e2da32762001f1 [id=67d7c591ac744a4c9d8249fad5a4709b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.667306 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 7c70286096b349ba9d17957df89b6601 (DEFAULT_TABLE table=default.Table_ed5a601ae75142dc9cc64766e464b377 [id=4d27c10582b74a2e8c8f49cca4bdde5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.669867 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e5cb3aaa4044eb9dd06227882e2b42" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:53.670292 5117 raft_consensus.cc:2243] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:53.670845 5117 raft_consensus.cc:2272] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:53.671077 4158 leader_election.cc:343] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:53.671536 4505 tablet_service.cc:1511] Processing CreateTablet for tablet d703e0b6c6cb433d9c49b5d7956ae360 (DEFAULT_TABLE table=default.Table_9cfed595445342da8d6b800243a1a693 [id=3be1c0abcd8c4027be6e1c0582c7c615]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:53.674926 4015 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet ebd991541273429e9c2bdc018ee402f5 (table default.Table_cf9ef037d1634978a0ddbe0a06f32284 [id=567cc975cfeb451087657471b5b225d3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ebd991541273429e9c2bdc018ee402f5 has no consensus state
I20260430 19:58:53.675786 5117 ts_tablet_manager.cc:1916] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:53.677688 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798bb6007af849e99bf8e95d84624044" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:53.678736 4158 leader_election.cc:343] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:53.679628 4509 tablet_service.cc:1511] Processing CreateTablet for tablet 5094639de3644f9ab7425d90f2e8d497 (DEFAULT_TABLE table=default.Table_0cc9e2f04597497a81fd4bf7d8da9734 [id=0d486775d9b547e1b951b107053434c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.684105 4742 tablet_replica.cc:333] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:53.685441 4742 raft_consensus.cc:2243] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:53.688902 4742 raft_consensus.cc:2272] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:53.691275 4742 ts_tablet_manager.cc:1916] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:53.690989 4508 tablet_service.cc:1511] Processing CreateTablet for tablet a66b8e3b2b56449e82f738342ff81f0d (DEFAULT_TABLE table=default.Table_d155c3e7331947ccb6d24c2244ec2537 [id=3b3dedc4c073491aa37913ef9852d573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.660771 4818 tablet_bootstrap.cc:492] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:53.695801 4818 ts_tablet_manager.cc:1403] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.271s user 0.012s sys 0.001s
I20260430 19:58:53.698385 4818 raft_consensus.cc:359] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.701347 4818 raft_consensus.cc:385] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.701683 4818 raft_consensus.cc:740] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.703508 4354 tablet_service.cc:1511] Processing CreateTablet for tablet 7c70286096b349ba9d17957df89b6601 (DEFAULT_TABLE table=default.Table_ed5a601ae75142dc9cc64766e464b377 [id=4d27c10582b74a2e8c8f49cca4bdde5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.704653 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c70286096b349ba9d17957df89b6601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.705768 4363 tablet_service.cc:1511] Processing CreateTablet for tablet a66b8e3b2b56449e82f738342ff81f0d (DEFAULT_TABLE table=default.Table_d155c3e7331947ccb6d24c2244ec2537 [id=3b3dedc4c073491aa37913ef9852d573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.706697 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a66b8e3b2b56449e82f738342ff81f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.706990 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 5094639de3644f9ab7425d90f2e8d497 (DEFAULT_TABLE table=default.Table_0cc9e2f04597497a81fd4bf7d8da9734 [id=0d486775d9b547e1b951b107053434c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.707284 4352 tablet_service.cc:1511] Processing CreateTablet for tablet d703e0b6c6cb433d9c49b5d7956ae360 (DEFAULT_TABLE table=default.Table_9cfed595445342da8d6b800243a1a693 [id=3be1c0abcd8c4027be6e1c0582c7c615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.707907 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5094639de3644f9ab7425d90f2e8d497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.708382 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d703e0b6c6cb433d9c49b5d7956ae360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.722527 4776 tablet_bootstrap.cc:492] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:53.722971 4776 ts_tablet_manager.cc:1403] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.104s user 0.010s sys 0.012s
I20260430 19:58:53.723663 4023 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5be52b4f74d74f8ca79ba986a2ab7334" } new_table_name: "default.Table_a02d418fcfd84af5b10f043467429963" modify_external_catalogs: true
I20260430 19:58:53.724496 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet a4b0d830f1b14ce382b2e0242f6565e7 (table default.Table_5cbfe354833844588121cc44d0a76e64 [id=d7de05cb84324624a2ebe601f5049040]) successfully deleted
I20260430 19:58:53.725206 4020 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet d4e34f3b1bf84f72855cf87683c5e318 (table default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519]). Expected version 1 got 0
I20260430 19:58:53.725282 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074]) successfully deleted
W20260430 19:58:53.729238 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet c81b11280f6d49e681537c88fb1cbf68 because the tablet was not found. No further retry: Not found: Tablet not found: c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:53.731906 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:53.734133 5092 leader_election.cc:290] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:53.737818 4507 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9dd03c317f4aaf9cb83e9429298981 (DEFAULT_TABLE table=default.Table_1543a16cba5a42498fd6f031d53ed6a1 [id=5853490dbc764e2ba7bfec2ec708657f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.743116 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9dd03c317f4aaf9cb83e9429298981 (DEFAULT_TABLE table=default.Table_1543a16cba5a42498fd6f031d53ed6a1 [id=5853490dbc764e2ba7bfec2ec708657f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.702412 4818 consensus_queue.cc:260] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.744172 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9dd03c317f4aaf9cb83e9429298981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.745165 4818 ts_tablet_manager.cc:1434] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.049s user 0.005s sys 0.000s
I20260430 19:58:53.745023 4729 raft_consensus.cc:359] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:53.745610 4729 raft_consensus.cc:385] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.745839 4729 raft_consensus.cc:740] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.745958 4818 tablet_bootstrap.cc:492] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:53.748720 5160 tablet.cc:1724] T c78d50690da7428fadb848dadefec47c P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260430 19:58:53.751516 5158 tablet.cc:1724] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:53.763103 4021 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 4758b3d846434b2b8466033c083b7339
I20260430 19:58:53.763648 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_0172fa4b93034ecfb61f52915a6abb8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:53.764268 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet f5c54c37bdae458d88594a3f5c401a48
W20260430 19:58:53.765362 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0172fa4b93034ecfb61f52915a6abb8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.768316 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet f5c54c37bdae458d88594a3f5c401a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:53.773934 4026 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet adf6f4bb3e16430a8a4b10dea4872c98 consensus state has no leader
I20260430 19:58:53.774246 4507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9dd03c317f4aaf9cb83e9429298981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.779532 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84b02b29777341d3934f6a77967d79b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.750149 5159 tablet.cc:1724] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:53.746239 4776 raft_consensus.cc:359] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:53.784368 4776 raft_consensus.cc:385] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.784713 4776 raft_consensus.cc:740] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.785521 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c70286096b349ba9d17957df89b6601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.785281 4776 consensus_queue.cc:260] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:53.787155 4776 ts_tablet_manager.cc:1434] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.064s user 0.005s sys 0.000s
I20260430 19:58:53.787940 4776 tablet_bootstrap.cc:492] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:53.791064 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d703e0b6c6cb433d9c49b5d7956ae360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.774443 4742 ts_tablet_manager.cc:1929] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:53.794746 4742 log.cc:1199] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/6220177b3e3e4d18bd8e20e6d9d96be7
I20260430 19:58:53.795665 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5094639de3644f9ab7425d90f2e8d497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.796986 4742 ts_tablet_manager.cc:1950] T 6220177b3e3e4d18bd8e20e6d9d96be7 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:53.798794 4776 tablet_bootstrap.cc:654] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.799223 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a66b8e3b2b56449e82f738342ff81f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.798467 4742 tablet_replica.cc:333] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:53.802203 4742 raft_consensus.cc:2243] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:53.802803 4742 raft_consensus.cc:2272] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:53.803022 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d36c2538ec84e0889c4e53975a2ff1c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:53.820837 4023 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a9ee8deb609742a3ae6f4f1c0b13308f consensus state has no leader
I20260430 19:58:53.820941 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet db0efaecfef44457a5128e878796c061
I20260430 19:58:53.822136 4025 catalog_manager.cc:6741] ReplaceTablet: tablet db0efaecfef44457a5128e878796c061 deleted and replaced by tablet d0c2c08fac7248a1b3b52e2b43041dff
I20260430 19:58:53.823318 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet db0efaecfef44457a5128e878796c061 with delete_type TABLET_DATA_DELETED (replaced by tablet d0c2c08fac7248a1b3b52e2b43041dff) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.859413 4026 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_41ec2a62305f427a9f2a581b7dce9c85 [id=94190b075a2a4a828681f5bb6594f8a2] alter complete (version 5)
I20260430 19:58:53.767247 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 4758b3d846434b2b8466033c083b7339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.769057 4351 tablet_service.cc:1558] Processing DeleteTablet for tablet f5c54c37bdae458d88594a3f5c401a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.833813 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet db0efaecfef44457a5128e878796c061 with delete_type TABLET_DATA_DELETED (replaced by tablet d0c2c08fac7248a1b3b52e2b43041dff) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.871650 4729 consensus_queue.cc:260] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:53.881644 4729 ts_tablet_manager.cc:1434] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.337s user 0.008s sys 0.000s
I20260430 19:58:53.846167 4025 catalog_manager.cc:5671] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2). New cstate: current_term: 1 leader_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:53.883280 4024 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:53.890460 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
I20260430 19:58:53.921720 4024 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:53.932842 4776 tablet_bootstrap.cc:492] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:53.933287 4776 ts_tablet_manager.cc:1403] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.146s user 0.032s sys 0.053s
I20260430 19:58:53.936062 4776 raft_consensus.cc:359] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:53.936647 4776 raft_consensus.cc:385] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:53.936905 4776 raft_consensus.cc:740] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:53.939877 4024 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_e298a596df714d4494a339954849530b [id=9713902a096f4953a4287b4078bea519] alter complete (version 1)
I20260430 19:58:53.944113 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e62335e43c1410b8b3acdd9e2f47981 with delete_type TABLET_DATA_DELETED (replaced by tablet cfd78a1cd7a94e27bb376d1b7c151073) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:53.944500 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 4758b3d846434b2b8466033c083b7339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:53.960319 4357 tablet_service.cc:1511] Processing CreateTablet for tablet b0d0ac18ef9040c8a5bd804178081083 (DEFAULT_TABLE table=default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.963349 4503 tablet_service.cc:1511] Processing CreateTablet for tablet b0d0ac18ef9040c8a5bd804178081083 (DEFAULT_TABLE table=default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.963493 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 239a2c6a534f4f478e436d14f94cd762 (DEFAULT_TABLE table=default.Table_cf1378d9973a48079c8e2b0bea84dc2a [id=51ad748be7c14d32ac99d6f8f63f569e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:53.964463 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d0ac18ef9040c8a5bd804178081083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:53.967679 4915 ts_tablet_manager.cc:1929] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:53.968031 4915 log.cc:1199] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/0e62335e43c1410b8b3acdd9e2f47981
I20260430 19:58:53.968737 4915 ts_tablet_manager.cc:1950] T 0e62335e43c1410b8b3acdd9e2f47981 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:53.971223 4818 tablet_bootstrap.cc:654] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:53.974061 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_f64798677d5443c394801041788a4e38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:53.974778 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ae4696042de446dbb711b9ce3ee33e3c" } modify_external_catalogs: true
I20260430 19:58:53.975270 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_ae4696042de446dbb711b9ce3ee33e3c" } modify_external_catalogs: true
W20260430 19:58:53.975559 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f64798677d5443c394801041788a4e38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:53.977346 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_1543a16cba5a42498fd6f031d53ed6a1" } new_table_name: "default.Table_4ca95d7064ab45feb5c75ce613e10122" modify_external_catalogs: true
I20260430 19:58:53.978335 4025 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_028cf8f724c64a51adefa0cc9c15a50e" } modify_external_catalogs: true
I20260430 19:58:53.978798 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_33d79062c8a6462ca2522c82cf2826b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:53.979301 4025 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_028cf8f724c64a51adefa0cc9c15a50e" } modify_external_catalogs: true
W20260430 19:58:53.980168 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33d79062c8a6462ca2522c82cf2826b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.002719 4508 tablet_service.cc:1511] Processing CreateTablet for tablet 239a2c6a534f4f478e436d14f94cd762 (DEFAULT_TABLE table=default.Table_cf1378d9973a48079c8e2b0bea84dc2a [id=51ad748be7c14d32ac99d6f8f63f569e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.003826 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239a2c6a534f4f478e436d14f94cd762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.003969 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d0ac18ef9040c8a5bd804178081083. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:54.008812 4027 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e9dd03c317f4aaf9cb83e9429298981 consensus state has no leader
I20260430 19:58:54.013155 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 33d13905b1144012b0e12039e9b3a087 (DEFAULT_TABLE table=default.Table_4e505a72d8d140029f5450a3d9572d4b [id=3ef0f9832e53421a9f81aa1371537e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.014242 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33d13905b1144012b0e12039e9b3a087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.014086 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 33d13905b1144012b0e12039e9b3a087 (DEFAULT_TABLE table=default.Table_4e505a72d8d140029f5450a3d9572d4b [id=3ef0f9832e53421a9f81aa1371537e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.015051 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33d13905b1144012b0e12039e9b3a087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.022843 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239a2c6a534f4f478e436d14f94cd762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.034850 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_7e4d0c964a2545d3b6ee8e7ea7e77ea0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:54.036485 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e4d0c964a2545d3b6ee8e7ea7e77ea0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.048322 4729 tablet_bootstrap.cc:492] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:54.050048 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_89f3aa5fc8b44b34b6bc2fe920db3e5e" } new_table_name: "default.Table_425e3062d49141638642c50ca47fd102" modify_external_catalogs: true
I20260430 19:58:54.050282 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 0e62335e43c1410b8b3acdd9e2f47981 (table default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620]) successfully deleted
W20260430 19:58:54.056177 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 51e35e0981e649f684caa96499040c15 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51e35e0981e649f684caa96499040c15 already in progress: creating tablet
W20260430 19:58:54.058174 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet f50116c63b0747099de2d234ca3faaac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f50116c63b0747099de2d234ca3faaac already in progress: creating tablet
W20260430 19:58:54.058835 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet f5c54c37bdae458d88594a3f5c401a48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5c54c37bdae458d88594a3f5c401a48 already in progress: creating tablet
I20260430 19:58:54.058911 4024 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 6cef0197c00b4ab7a11e2e5123a4c277 (table default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483]). Expected version 1 got 0
W20260430 19:58:54.059432 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet db0efaecfef44457a5128e878796c061 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db0efaecfef44457a5128e878796c061 already in progress: creating tablet
W20260430 19:58:54.060005 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 4758b3d846434b2b8466033c083b7339 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4758b3d846434b2b8466033c083b7339 already in progress: creating tablet
I20260430 19:58:54.063179 4729 tablet_bootstrap.cc:654] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.063632 4581 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:54.064129 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:54.069736 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_d55e71cfbdcd48b280a28b690222278a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:54.071225 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d55e71cfbdcd48b280a28b690222278a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.071563 5117 ts_tablet_manager.cc:1929] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260430 19:58:54.072028 5117 log.cc:1199] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/fa59394fc34f419ca22a695b8dff4613
I20260430 19:58:54.072784 5117 ts_tablet_manager.cc:1950] T fa59394fc34f419ca22a695b8dff4613 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:54.077358 4028 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 499f1bc9d55849ca97fb4732ba240fee
I20260430 19:58:54.077971 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 34a6d662493e428aa475979abbffc9d7
I20260430 19:58:54.078738 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 153) took 1039 ms. Trace:
I20260430 19:58:54.079144 5117 rpcz_store.cc:276] 0430 19:58:53.039388 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.039515 (+ 127us) service_pool.cc:225] Handling call
0430 19:58:53.659020 (+619505us) ts_tablet_manager.cc:1151] Deleting tablet fa59394fc34f419ca22a695b8dff4613
0430 19:58:53.659068 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:54.031076 (+372008us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.071432 (+ 40356us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.078675 (+ 7243us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":618880}
I20260430 19:58:54.080083 4507 tablet_service.cc:1558] Processing DeleteTablet for tablet 34a6d662493e428aa475979abbffc9d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:54.081629 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 34a6d662493e428aa475979abbffc9d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34a6d662493e428aa475979abbffc9d7 already in progress: creating tablet
I20260430 19:58:54.082284 5117 tablet_replica.cc:333] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:54.086172 4352 tablet_service.cc:1558] Processing DeleteTablet for tablet 34a6d662493e428aa475979abbffc9d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:54.086831 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074]) successfully deleted
I20260430 19:58:54.087461 5117 raft_consensus.cc:2243] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:54.087960 5117 raft_consensus.cc:2272] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:54.088074 4729 tablet_bootstrap.cc:492] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:54.088539 4729 ts_tablet_manager.cc:1403] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.000s
I20260430 19:58:54.089962 5117 ts_tablet_manager.cc:1916] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.090991 4729 raft_consensus.cc:359] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.092231 4818 tablet_bootstrap.cc:492] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:54.092535 4729 raft_consensus.cc:385] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.092651 4818 ts_tablet_manager.cc:1403] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.347s user 0.008s sys 0.000s
I20260430 19:58:54.094906 4729 raft_consensus.cc:740] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:54.095336 5092 raft_consensus.cc:493] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.095815 5092 raft_consensus.cc:515] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.095755 4729 consensus_queue.cc:260] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:54.099416 4317 leader_election.cc:343] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:54.100708 5092 leader_election.cc:290] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:54.103500 4729 ts_tablet_manager.cc:1434] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260430 19:58:54.104408 4729 tablet_bootstrap.cc:492] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:54.126533 4381 raft_consensus.cc:3060] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:54.127815 5117 ts_tablet_manager.cc:1929] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:54.128219 5117 log.cc:1199] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/2c119dd9a5d944eabd588f62a76da98e
I20260430 19:58:54.129091 5117 ts_tablet_manager.cc:1950] T 2c119dd9a5d944eabd588f62a76da98e P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:54.130762 4729 tablet_bootstrap.cc:654] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.130942 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e5cb3aaa4044eb9dd06227882e2b42" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:54.131232 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 155) took 1003 ms. Trace:
I20260430 19:58:54.131031 4818 raft_consensus.cc:359] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.131676 4818 raft_consensus.cc:385] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.132009 4818 raft_consensus.cc:740] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:54.132148 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798bb6007af849e99bf8e95d84624044" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:54.132606 4818 consensus_queue.cc:260] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.131536 5117 rpcz_store.cc:276] 0430 19:58:53.127850 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.127994 (+ 144us) service_pool.cc:225] Handling call
0430 19:58:54.082114 (+954120us) ts_tablet_manager.cc:1151] Deleting tablet 2c119dd9a5d944eabd588f62a76da98e
0430 19:58:54.082165 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:54.113741 (+ 31576us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.121649 (+ 7908us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.131162 (+ 9513us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":952005}
I20260430 19:58:54.139146 5117 tablet_replica.cc:333] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:54.140254 5117 raft_consensus.cc:2243] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
W20260430 19:58:54.140075 4161 leader_election.cc:343] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:54.140650 4161 leader_election.cc:304] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:54.144075 4818 ts_tablet_manager.cc:1434] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.051s user 0.006s sys 0.000s
I20260430 19:58:54.155632 5117 raft_consensus.cc:2272] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:54.157807 5117 ts_tablet_manager.cc:1916] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.163067 4729 tablet_bootstrap.cc:492] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:54.163439 4729 ts_tablet_manager.cc:1403] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.059s user 0.006s sys 0.003s
I20260430 19:58:54.165642 4729 raft_consensus.cc:359] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.166225 4729 raft_consensus.cc:385] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.166461 4729 raft_consensus.cc:740] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:54.167030 4729 consensus_queue.cc:260] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.168834 4729 ts_tablet_manager.cc:1434] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260430 19:58:54.172000 4818 tablet_bootstrap.cc:492] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:54.175164 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 2c119dd9a5d944eabd588f62a76da98e (table default.Table_cf1378d9973a48079c8e2b0bea84dc2a [id=51ad748be7c14d32ac99d6f8f63f569e]) successfully deleted
I20260430 19:58:54.176785 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 6220177b3e3e4d18bd8e20e6d9d96be7 (table default.Table_0a03c960230b4206944b333b146c39c7 [id=26c468440dce47839c5c48e7bc76c173]) successfully deleted
W20260430 19:58:54.177361 4016 catalog_manager.cc:4969] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 09f272340cd942e9b2af9e38a2bdec82 because the tablet was not found. No further retry: Not found: Tablet not found: 09f272340cd942e9b2af9e38a2bdec82
I20260430 19:58:54.181466 4729 tablet_bootstrap.cc:492] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
W20260430 19:58:54.187588 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 798bb6007af849e99bf8e95d84624044 consensus state has no leader
I20260430 19:58:54.191133 4189 tablet_service.cc:1511] Processing CreateTablet for tablet 33d13905b1144012b0e12039e9b3a087 (DEFAULT_TABLE table=default.Table_4e505a72d8d140029f5450a3d9572d4b [id=3ef0f9832e53421a9f81aa1371537e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.192230 4191 tablet_service.cc:1511] Processing CreateTablet for tablet 239a2c6a534f4f478e436d14f94cd762 (DEFAULT_TABLE table=default.Table_cf1378d9973a48079c8e2b0bea84dc2a [id=51ad748be7c14d32ac99d6f8f63f569e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.192968 4189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33d13905b1144012b0e12039e9b3a087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.194029 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 5094639de3644f9ab7425d90f2e8d497 (DEFAULT_TABLE table=default.Table_0cc9e2f04597497a81fd4bf7d8da9734 [id=0d486775d9b547e1b951b107053434c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.195046 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5094639de3644f9ab7425d90f2e8d497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.191133 4190 tablet_service.cc:1511] Processing CreateTablet for tablet 7c70286096b349ba9d17957df89b6601 (DEFAULT_TABLE table=default.Table_ed5a601ae75142dc9cc64766e464b377 [id=4d27c10582b74a2e8c8f49cca4bdde5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.195902 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c70286096b349ba9d17957df89b6601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.198529 4191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239a2c6a534f4f478e436d14f94cd762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.206220 4818 tablet_bootstrap.cc:654] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.212363 4204 tablet_service.cc:1558] Processing DeleteTablet for tablet 70d410f3b3b04e649b79dda04c0891ee with delete_type TABLET_DATA_DELETED (replaced by tablet b0d0ac18ef9040c8a5bd804178081083) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.213202 4193 tablet_service.cc:1511] Processing CreateTablet for tablet a66b8e3b2b56449e82f738342ff81f0d (DEFAULT_TABLE table=default.Table_d155c3e7331947ccb6d24c2244ec2537 [id=3b3dedc4c073491aa37913ef9852d573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.214762 4274 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:54.219061 5155 raft_consensus.cc:493] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.221817 4196 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9dd03c317f4aaf9cb83e9429298981 (DEFAULT_TABLE table=default.Table_1543a16cba5a42498fd6f031d53ed6a1 [id=5853490dbc764e2ba7bfec2ec708657f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.222988 4024 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_36c4be64f98844e591eac49478ce90d5" } new_table_name: "default.Table_fd1d6bc4a69a43dda920f1320e3412b7" modify_external_catalogs: true
I20260430 19:58:53.937414 4776 consensus_queue.cc:260] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.186439 4729 tablet_bootstrap.cc:654] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.254951 4199 tablet_service.cc:1558] Processing DeleteTablet for tablet db0efaecfef44457a5128e878796c061 with delete_type TABLET_DATA_DELETED (replaced by tablet d0c2c08fac7248a1b3b52e2b43041dff) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.255642 5155 raft_consensus.cc:515] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:54.258044 5155 leader_election.cc:290] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:54.264477 5092 raft_consensus.cc:493] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.264914 5092 raft_consensus.cc:515] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.276903 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_16a0661d7f5f41c18b86be9745bc54a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:54.278314 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16a0661d7f5f41c18b86be9745bc54a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.280534 4025 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e" } modify_external_catalogs: true
I20260430 19:58:54.280994 4025 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e" } modify_external_catalogs: true
I20260430 19:58:54.259855 4198 tablet_service.cc:1511] Processing CreateTablet for tablet b0d0ac18ef9040c8a5bd804178081083 (DEFAULT_TABLE table=default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.282552 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a66b8e3b2b56449e82f738342ff81f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.283596 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_cc310773560b43f89ee4742daa2360c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:54.286573 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_1ef1c0b4ac9f4d5ca9433f755ecf83fe" } modify_external_catalogs: true
I20260430 19:58:54.287071 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_1ef1c0b4ac9f4d5ca9433f755ecf83fe" } modify_external_catalogs: true
I20260430 19:58:54.289862 4198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d0ac18ef9040c8a5bd804178081083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.261392 4201 tablet_service.cc:1558] Processing DeleteTablet for tablet 4758b3d846434b2b8466033c083b7339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:41996
W20260430 19:58:54.291468 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc310773560b43f89ee4742daa2360c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.260273 4203 tablet_service.cc:1511] Processing CreateTablet for tablet d703e0b6c6cb433d9c49b5d7956ae360 (DEFAULT_TABLE table=default.Table_9cfed595445342da8d6b800243a1a693 [id=3be1c0abcd8c4027be6e1c0582c7c615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.265635 4200 tablet_service.cc:1558] Processing DeleteTablet for tablet 34a6d662493e428aa475979abbffc9d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.294759 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d703e0b6c6cb433d9c49b5d7956ae360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.296119 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745f04d04091443ca1c39949d005dd69" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:54.260535 4197 tablet_service.cc:1558] Processing DeleteTablet for tablet f5c54c37bdae458d88594a3f5c401a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.307689 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9dd03c317f4aaf9cb83e9429298981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.258569 5155 raft_consensus.cc:2749] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:54.259639 4960 raft_consensus.cc:493] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.320156 4960 raft_consensus.cc:515] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.321919 4960 leader_election.cc:290] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:54.224593 5135 raft_consensus.cc:493] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.200492 4508 tablet_service.cc:1558] Processing DeleteTablet for tablet 51e35e0981e649f684caa96499040c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:54.282003 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:54.336216 5135 raft_consensus.cc:515] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:54.336792 4915 tablet_replica.cc:333] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:54.337510 4915 raft_consensus.cc:2243] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:54.337946 4915 raft_consensus.cc:2272] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:54.338249 5135 leader_election.cc:290] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:54.338796 4531 raft_consensus.cc:2468] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
I20260430 19:58:54.323663 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:54.342980 4915 ts_tablet_manager.cc:1916] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.343967 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet 09f272340cd942e9b2af9e38a2bdec82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:54.346179 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet 404f20adf9d741f3a036b06cbdbaedb8 from {username='slave'} at 127.0.0.1:55130
W20260430 19:58:54.348511 4024 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet adf6f4bb3e16430a8a4b10dea4872c98 consensus state has no leader
I20260430 19:58:54.343451 5092 leader_election.cc:290] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:54.350569 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet eafc058f392a45de9d30b521ebde0950
W20260430 19:58:54.351802 4317 leader_election.cc:343] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 0e62335e43c1410b8b3acdd9e2f47981
I20260430 19:58:54.353018 4508 tablet_service.cc:1558] Processing DeleteTablet for tablet eafc058f392a45de9d30b521ebde0950 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:54.353199 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:54.354470 4470 leader_election.cc:343] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:54.360735 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet eafc058f392a45de9d30b521ebde0950 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:54.364190 4915 ts_tablet_manager.cc:1929] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:54.368949 4915 log.cc:1199] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/51e35e0981e649f684caa96499040c15
I20260430 19:58:54.369707 4915 ts_tablet_manager.cc:1950] T 51e35e0981e649f684caa96499040c15 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:54.371093 4742 ts_tablet_manager.cc:1916] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.372087 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 51e35e0981e649f684caa96499040c15 (table default.Table_4f04fa211ec3447488b05b4f5b157487 [id=56ead7419d654e5dad1631c264ec87a5]) successfully deleted
I20260430 19:58:54.372982 4915 tablet_replica.cc:333] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:54.373721 4915 raft_consensus.cc:2243] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:54.377676 4818 tablet_bootstrap.cc:492] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:54.378098 4818 ts_tablet_manager.cc:1403] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.206s user 0.006s sys 0.002s
I20260430 19:58:54.378206 4915 raft_consensus.cc:2272] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:54.379971 4915 ts_tablet_manager.cc:1916] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.381822 5117 ts_tablet_manager.cc:1929] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:54.386246 5117 log.cc:1199] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:54.388305 5117 ts_tablet_manager.cc:1950] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:54.385560 4818 raft_consensus.cc:359] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.389046 4818 raft_consensus.cc:385] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.389978 4818 raft_consensus.cc:740] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:54.396885 4729 tablet_bootstrap.cc:492] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:54.397541 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_5b5580edb7844ccb8271ad24e96a1eb0" } new_table_name: "default.Table_a24540b6ec7948578c87d89d6116fdb9" modify_external_catalogs: true
I20260430 19:58:54.397679 4818 consensus_queue.cc:260] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.397023 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 158) took 1093 ms. Trace:
I20260430 19:58:54.398768 5117 rpcz_store.cc:276] 0430 19:58:53.303213 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.303994 (+ 781us) service_pool.cc:225] Handling call
0430 19:58:54.138932 (+834938us) ts_tablet_manager.cc:1151] Deleting tablet c81b11280f6d49e681537c88fb1cbf68
0430 19:58:54.138972 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:54.233754 (+ 94782us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.381701 (+147947us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.396958 (+ 15257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14713,"tablet-delete.queue_time_us":834377}
I20260430 19:58:54.399948 5117 tablet_replica.cc:333] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
W20260430 19:58:54.401190 4320 leader_election.cc:343] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: c81b11280f6d49e681537c88fb1cbf68
I20260430 19:58:54.401554 4320 leader_election.cc:304] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:54.401854 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet c81b11280f6d49e681537c88fb1cbf68 (table default.Table_f79093a8060a4af3979bd0705d8a6998 [id=7b43b6a692214e0b89c73b9374682197]) successfully deleted
W20260430 19:58:54.402170 4320 raft_consensus.cc:2686] T c81b11280f6d49e681537c88fb1cbf68 P d8345ec7f57c4399a8a4afe4dd43439e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260430 19:58:54.397287 4729 ts_tablet_manager.cc:1403] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.216s user 0.010s sys 0.002s
I20260430 19:58:54.411433 4320 leader_election.cc:304] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:54.413151 5092 raft_consensus.cc:2749] T 745f04d04091443ca1c39949d005dd69 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:54.419360 4027 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 798bb6007af849e99bf8e95d84624044 (table default.Table_425e3062d49141638642c50ca47fd102 [id=d963a034463343989c1d699cce2134a5]). Expected version 1 got 0
I20260430 19:58:54.420274 4158 leader_election.cc:304] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:54.421202 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet eafc058f392a45de9d30b521ebde0950 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.421857 5165 raft_consensus.cc:2804] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:54.422190 5165 raft_consensus.cc:493] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:54.422461 5165 raft_consensus.cc:3060] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:54.423568 4158 leader_election.cc:343] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:54.423660 4729 raft_consensus.cc:359] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.428437 4742 ts_tablet_manager.cc:1929] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:54.428869 4742 log.cc:1199] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/404f20adf9d741f3a036b06cbdbaedb8
I20260430 19:58:54.429647 4742 ts_tablet_manager.cc:1950] T 404f20adf9d741f3a036b06cbdbaedb8 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:54.429481 4508 tablet_service.cc:1511] Processing CreateTablet for tablet a0e0c3841ebd449c9ac742432ac3a144 (DEFAULT_TABLE table=default.Table_50723dc3e29240dbba36ca94e7b72a3f [id=8a110936ef1543b198f6c769d57e64f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.424291 4729 raft_consensus.cc:385] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.430320 4729 raft_consensus.cc:740] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:54.430630 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e0c3841ebd449c9ac742432ac3a144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.430938 4729 consensus_queue.cc:260] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.433173 4360 tablet_service.cc:1511] Processing CreateTablet for tablet a0e0c3841ebd449c9ac742432ac3a144 (DEFAULT_TABLE table=default.Table_50723dc3e29240dbba36ca94e7b72a3f [id=8a110936ef1543b198f6c769d57e64f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.433903 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 93408323994642129d282ecfdd964070 (DEFAULT_TABLE table=default.Table_5ff207a6e7ed439692d024dfc07accae [id=05a98ab830ee42c1ab747641465c3e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.434124 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e0c3841ebd449c9ac742432ac3a144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.434806 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93408323994642129d282ecfdd964070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.436185 5135 raft_consensus.cc:493] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.436578 5135 raft_consensus.cc:515] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.438938 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 51e35e0981e649f684caa96499040c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:54.439040 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798bb6007af849e99bf8e95d84624044" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:54.440846 4470 leader_election.cc:343] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:54.441066 4021 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:54.441555 5135 leader_election.cc:290] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:54.446499 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e62335e43c1410b8b3acdd9e2f47981" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:54.447098 4223 raft_consensus.cc:2468] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:54.450163 4581 heartbeater.cc:499] Master 127.3.190.62:33781 was elected leader, sending a full tablet report...
I20260430 19:58:54.455343 5165 raft_consensus.cc:515] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:54.458182 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:54.458477 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 93408323994642129d282ecfdd964070 (DEFAULT_TABLE table=default.Table_5ff207a6e7ed439692d024dfc07accae [id=05a98ab830ee42c1ab747641465c3e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.458823 4530 raft_consensus.cc:3060] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:54.459499 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93408323994642129d282ecfdd964070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.463785 4503 tablet_service.cc:1511] Processing CreateTablet for tablet a590ba6ee85d48d2a329b1ab2efd364c (DEFAULT_TABLE table=default.Table_6ca128f6264545baa6d5b0dfe8023a8e [id=87bf981be4104970ae06aadb09b7f160]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.464740 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a590ba6ee85d48d2a329b1ab2efd364c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:54.465590 4317 consensus_peers.cc:597] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6220177b3e3e4d18bd8e20e6d9d96be7. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:54.466032 4818 ts_tablet_manager.cc:1434] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.084s user 0.001s sys 0.005s
W20260430 19:58:54.468327 5145 lock_manager.cc:585] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 6180614032372641643) tablet 00000000000000000000000000000000 cur holder 0x7b5800051000
I20260430 19:58:54.474790 4818 tablet_bootstrap.cc:492] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:54.400743 5117 raft_consensus.cc:2243] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:54.478971 5117 raft_consensus.cc:2272] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:54.478989 4161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 339205us
W20260430 19:58:54.479646 4161 leader_election.cc:343] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:54.480010 4161 leader_election.cc:304] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:54.480733 5166 raft_consensus.cc:2749] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:54.481236 4320 leader_election.cc:304] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:54.482367 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 159) took 1203 ms. Trace:
I20260430 19:58:54.483067 5092 raft_consensus.cc:2804] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:54.483397 5092 raft_consensus.cc:493] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:54.483707 5092 raft_consensus.cc:3060] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:54.486413 4818 tablet_bootstrap.cc:654] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.488220 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:54.492038 5092 raft_consensus.cc:515] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.494401 4530 raft_consensus.cc:2468] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:54.485879 5166 consensus_queue.cc:1048] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:54.500965 4507 tablet_service.cc:1511] Processing CreateTablet for tablet ec7ea59485f046919b87e9fcaf8686ca (DEFAULT_TABLE table=default.Table_8a76e0b1a42d4a27b12f435c0fef102b [id=1f5cd2df607e41cbb581c910ca5e02f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.500831 4742 rpcz_store.cc:276] 0430 19:58:53.278459 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.278566 (+ 107us) service_pool.cc:225] Handling call
0430 19:58:53.798343 (+519777us) ts_tablet_manager.cc:1151] Deleting tablet 404f20adf9d741f3a036b06cbdbaedb8
0430 19:58:53.798373 (+ 30us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:54.390620 (+592247us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.428320 (+ 37700us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.482306 (+ 53986us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":519265}
I20260430 19:58:54.502081 4507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7ea59485f046919b87e9fcaf8686ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.502100 4742 tablet_replica.cc:333] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:54.502825 4742 raft_consensus.cc:2243] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:54.503271 4742 raft_consensus.cc:2272] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:54.505187 4158 leader_election.cc:304] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:54.505857 5166 raft_consensus.cc:2804] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:54.506407 5166 raft_consensus.cc:697] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:54.507376 4191 tablet_service.cc:1511] Processing CreateTablet for tablet a0e0c3841ebd449c9ac742432ac3a144 (DEFAULT_TABLE table=default.Table_50723dc3e29240dbba36ca94e7b72a3f [id=8a110936ef1543b198f6c769d57e64f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.507334 5166 consensus_queue.cc:237] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:54.508450 4191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e0c3841ebd449c9ac742432ac3a144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.511647 4729 ts_tablet_manager.cc:1434] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.103s user 0.005s sys 0.000s
I20260430 19:58:54.512491 4729 tablet_bootstrap.cc:492] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:54.515324 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_88431e83ce7e468aa602238e637bfa58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:54.516431 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_3ca34fb096c24aa692126c78e7d66dbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:54.516816 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88431e83ce7e468aa602238e637bfa58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.517985 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 404f20adf9d741f3a036b06cbdbaedb8 (table default.Table_2afe6c2b4071491f868a3a17cc4d6e48 [id=06859fba4a5d4b8f978e6f56edc5fb85]) successfully deleted
W20260430 19:58:54.522398 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ca34fb096c24aa692126c78e7d66dbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:54.488313 4320 consensus_peers.cc:597] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6220177b3e3e4d18bd8e20e6d9d96be7. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:54.505671 5117 ts_tablet_manager.cc:1916] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.516805 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:54.520535 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:54.528070 4358 tablet_service.cc:1558] Processing DeleteTablet for tablet e4e87fda401047cfa7070ae95b1701a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:47 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:54.531028 5092 leader_election.cc:290] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
W20260430 19:58:54.531343 4317 leader_election.cc:343] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 0e62335e43c1410b8b3acdd9e2f47981
I20260430 19:58:54.532095 4190 tablet_service.cc:1511] Processing CreateTablet for tablet 93408323994642129d282ecfdd964070 (DEFAULT_TABLE table=default.Table_5ff207a6e7ed439692d024dfc07accae [id=05a98ab830ee42c1ab747641465c3e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.532351 4776 ts_tablet_manager.cc:1434] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.599s user 0.022s sys 0.009s
I20260430 19:58:54.400509 4915 ts_tablet_manager.cc:1929] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260430 19:58:54.532796 4776 ts_tablet_manager.cc:1467] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: Tablet startup took 599ms
I20260430 19:58:54.533099 4915 log.cc:1199] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/eafc058f392a45de9d30b521ebde0950
I20260430 19:58:54.533191 4190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93408323994642129d282ecfdd964070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.533820 4915 ts_tablet_manager.cc:1950] T eafc058f392a45de9d30b521ebde0950 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:54.535138 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet eafc058f392a45de9d30b521ebde0950 (table default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e [id=8d1c3bb61831442596e2112cef9032a8]) successfully deleted
I20260430 19:58:54.539588 4509 tablet_service.cc:1511] Processing CreateTablet for tablet d272d84e3f3e4ff4ace102ac859426f6 (DEFAULT_TABLE table=default.Table_0c0f3cc776e847188b388e9ba7cb1285 [id=3b5da9bc474a44f896fce6cefddba90d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.540690 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d272d84e3f3e4ff4ace102ac859426f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.551614 4024 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_871afe5ef54a402c9d42abd2f1888527" } new_table_name: "default.Table_3a24adb795934ac7a893366f60e21964" modify_external_catalogs: true
I20260430 19:58:54.556473 4022 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_60d3a33c8bfc4641ac60f0ea151c1225" } modify_external_catalogs: true
I20260430 19:58:54.556967 4022 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_60d3a33c8bfc4641ac60f0ea151c1225" } modify_external_catalogs: true
I20260430 19:58:54.562255 4818 tablet_bootstrap.cc:492] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:54.562827 4818 ts_tablet_manager.cc:1403] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.006s
I20260430 19:58:54.566437 4818 raft_consensus.cc:359] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.569679 4818 raft_consensus.cc:385] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.571859 4818 raft_consensus.cc:740] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
W20260430 19:58:54.572952 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 51e35e0981e649f684caa96499040c15 because the tablet was not found. No further retry: Not found: Tablet not found: 51e35e0981e649f684caa96499040c15
I20260430 19:58:54.572652 4818 consensus_queue.cc:260] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.587457 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_13e7445cfe4d4ed9ae643ac477a76530"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:54.588912 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13e7445cfe4d4ed9ae643ac477a76530 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.590291 4508 tablet_service.cc:1511] Processing CreateTablet for tablet 1fd70a5179dc4436bb2ba5a04f54f819 (DEFAULT_TABLE table=default.Table_4ee70552d7224ff9a90e149ed7a17de4 [id=735a21065fcf4d8e9952b3716f72a43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.592837 4818 ts_tablet_manager.cc:1434] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260430 19:58:54.601812 4503 tablet_service.cc:1511] Processing CreateTablet for tablet 33ebbb7e29724e8191209283085059f5 (DEFAULT_TABLE table=default.Table_a2d74c305d5f4d8ca3db561150380152 [id=a329d4d27b4f4266a37db9e021a05fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.602963 4503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33ebbb7e29724e8191209283085059f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.604192 4508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd70a5179dc4436bb2ba5a04f54f819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.607133 4818 tablet_bootstrap.cc:492] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:54.600335 5117 ts_tablet_manager.cc:1929] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:54.613986 5117 log.cc:1199] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/37d6c54ae89946158cad45e741c1c0f0
I20260430 19:58:54.614785 4358 tablet_service.cc:1511] Processing CreateTablet for tablet ec7ea59485f046919b87e9fcaf8686ca (DEFAULT_TABLE table=default.Table_8a76e0b1a42d4a27b12f435c0fef102b [id=1f5cd2df607e41cbb581c910ca5e02f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.618918 4357 tablet_service.cc:1511] Processing CreateTablet for tablet a590ba6ee85d48d2a329b1ab2efd364c (DEFAULT_TABLE table=default.Table_6ca128f6264545baa6d5b0dfe8023a8e [id=87bf981be4104970ae06aadb09b7f160]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.534686 5165 leader_election.cc:290] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:54.578269 4198 tablet_service.cc:1558] Processing DeleteTablet for tablet 081a060c44924bcda09ffc8fe88012b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.621134 4352 tablet_service.cc:1511] Processing CreateTablet for tablet 1fd70a5179dc4436bb2ba5a04f54f819 (DEFAULT_TABLE table=default.Table_4ee70552d7224ff9a90e149ed7a17de4 [id=735a21065fcf4d8e9952b3716f72a43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.505322 4742 ts_tablet_manager.cc:1916] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.625708 4505 tablet_service.cc:1511] Processing CreateTablet for tablet fb145e6d80f64bdb99965a3cf1e889f7 (DEFAULT_TABLE table=default.Table_a9a78f62539447b98602897b63dd48f8 [id=8b29d1f3dd5d4b45bcd3051cd989c8ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.626860 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb145e6d80f64bdb99965a3cf1e889f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.627064 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_3e3a927708124dbfa583240f38fdbe55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:54.615825 4360 tablet_service.cc:1511] Processing CreateTablet for tablet d272d84e3f3e4ff4ace102ac859426f6 (DEFAULT_TABLE table=default.Table_0c0f3cc776e847188b388e9ba7cb1285 [id=3b5da9bc474a44f896fce6cefddba90d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:54.628607 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e3a927708124dbfa583240f38fdbe55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:54.629273 5117 ts_tablet_manager.cc:1950] T 37d6c54ae89946158cad45e741c1c0f0 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:54.637579 4818 tablet_bootstrap.cc:654] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.634956 4507 tablet_service.cc:1511] Processing CreateTablet for tablet 4219d92a298146ff8181728e1a95a0b7 (DEFAULT_TABLE table=default.Table_149dd50933e148538c23f32cf19ca2dd [id=fe0cd0ba31e44b6e9023e4cf663f42e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.640024 4507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4219d92a298146ff8181728e1a95a0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.645547 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7ea59485f046919b87e9fcaf8686ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.646953 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd70a5179dc4436bb2ba5a04f54f819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.652735 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a590ba6ee85d48d2a329b1ab2efd364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.654740 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d272d84e3f3e4ff4ace102ac859426f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.660401 4729 tablet_bootstrap.cc:654] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.667820 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 159) took 1345 ms. Trace:
I20260430 19:58:54.668268 5117 rpcz_store.cc:276] 0430 19:58:53.322198 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.322300 (+ 102us) service_pool.cc:225] Handling call
0430 19:58:54.399794 (+1077494us) ts_tablet_manager.cc:1151] Deleting tablet 37d6c54ae89946158cad45e741c1c0f0
0430 19:58:54.399835 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:54.558256 (+158421us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.600206 (+ 41950us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.642733 (+ 42527us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800000098. stack: 00007ff08b6a8c83 00007ff08b6a8b80 00007ff08b219574 000000000050e094 000000000050e029 00007ff08b518941 00007ff08b69d6c1 00007ff08b69dcf2 00007ff09450a469 00007ff09374a488 00007ff093811e12 00007ff093842990 00007ff093828efd 00007ff093828ea9 00007ff093828e39 00007ff093828e01
0430 19:58:54.667756 (+ 25023us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24301184,"tablet-delete.queue_time_us":1076958}
I20260430 19:58:54.670351 5117 tablet_replica.cc:333] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:54.671244 5117 raft_consensus.cc:2243] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:54.674829 4026 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet ea907f9bd93041de976a2f0c3b517dab
W20260430 19:58:54.675132 4028 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66e94750de27416e9adcf60fccf2ace9 consensus state has no leader
I20260430 19:58:54.680394 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 37d6c54ae89946158cad45e741c1c0f0 (table default.Table_871afe5ef54a402c9d42abd2f1888527 [id=e31eb8776ad0452d91a6465c3c79253c]) successfully deleted
I20260430 19:58:54.682834 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet ea907f9bd93041de976a2f0c3b517dab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:54.684230 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet ea907f9bd93041de976a2f0c3b517dab because the tablet was not found. No further retry: Not found: Tablet not found: ea907f9bd93041de976a2f0c3b517dab
W20260430 19:58:54.684633 4014 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet cfd78a1cd7a94e27bb376d1b7c151073 (table default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cfd78a1cd7a94e27bb376d1b7c151073 consensus state has no leader
I20260430 19:58:54.682083 4354 tablet_service.cc:1511] Processing CreateTablet for tablet 33ebbb7e29724e8191209283085059f5 (DEFAULT_TABLE table=default.Table_a2d74c305d5f4d8ca3db561150380152 [id=a329d4d27b4f4266a37db9e021a05fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.689428 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33ebbb7e29724e8191209283085059f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.692919 4509 tablet_service.cc:1511] Processing CreateTablet for tablet 5d405ddf1037400f9d87dda4c3b0e911 (DEFAULT_TABLE table=default.Table_a73bf5f11b0e46198f70577bc06dcf91 [id=ce2e2befe1a34e7eb0a693f48cf369c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.683965 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 4219d92a298146ff8181728e1a95a0b7 (DEFAULT_TABLE table=default.Table_149dd50933e148538c23f32cf19ca2dd [id=fe0cd0ba31e44b6e9023e4cf663f42e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.694011 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d405ddf1037400f9d87dda4c3b0e911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.694124 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4219d92a298146ff8181728e1a95a0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.695899 5187 tablet.cc:1724] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:54.707559 5117 pending_rounds.cc:70] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:54.710117 5117 pending_rounds.cc:77] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964129766998016 op_type: NO_OP noop_request { }
I20260430 19:58:54.710789 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 3ea24c4cdb4d43aba5762f0073488535 (DEFAULT_TABLE table=default.Table_4472ad7bad434d159c5275dc870c286c [id=51d4b8ce92574939bdc73ecd7c32d080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.675200 4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
W20260430 19:58:54.533070 4776 ts_tablet_manager.cc:1469] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: Trace:
0430 19:58:48.511782 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:48.518697 (+ 6915us) service_pool.cc:225] Handling call
0430 19:58:48.519911 (+ 1214us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0430 19:58:48.519996 (+ 85us) ts_tablet_manager.cc:682] Creating new metadata...
0430 19:58:48.570813 (+ 50817us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:48.714404 (+143591us) tablet_replica.cc:206] Creating consensus instance
0430 19:58:48.714853 (+ 449us) inbound_call.cc:177] Queueing success response
0430 19:58:53.787677 (+5072824us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0430 19:58:53.933525 (+145848us) ts_tablet_manager.cc:1435] Starting tablet replica
0430 19:58:53.934527 (+ 1002us) tablet_replica.cc:261] Starting instrumentation
0430 19:58:53.934959 (+ 432us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":91796,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":246,"tablet-open.queue_time_us":5072920,"thread_start_us":170521,"threads_started":1}
W20260430 19:58:54.712412 5172 log.cc:927] Time spent T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: Append to log took a long time: real 0.195s user 0.015s sys 0.005s
I20260430 19:58:54.713526 4776 tablet_bootstrap.cc:492] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:54.714109 4189 tablet_service.cc:1511] Processing CreateTablet for tablet ec7ea59485f046919b87e9fcaf8686ca (DEFAULT_TABLE table=default.Table_8a76e0b1a42d4a27b12f435c0fef102b [id=1f5cd2df607e41cbb581c910ca5e02f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.731590 4196 tablet_service.cc:1558] Processing DeleteTablet for tablet b6acec54bc7b4692894333e0150a1aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.738564 4198 tablet_service.cc:1511] Processing CreateTablet for tablet a590ba6ee85d48d2a329b1ab2efd364c (DEFAULT_TABLE table=default.Table_6ca128f6264545baa6d5b0dfe8023a8e [id=87bf981be4104970ae06aadb09b7f160]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.739915 4198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a590ba6ee85d48d2a329b1ab2efd364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.731380 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e62335e43c1410b8b3acdd9e2f47981" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:54.685672 5186 tablet.cc:1724] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260430 19:58:54.743851 4320 leader_election.cc:343] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:58:54.744443 4320 leader_election.cc:304] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:54.744611 4502 tablet_service.cc:1511] Processing CreateTablet for tablet 7cb31c7051734e79966684015e05e631 (DEFAULT_TABLE table=default.Table_968941fa27b84dfab105dbb83df734ab [id=aa554293152a489cb6f3ad7fb3f67d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.746331 5092 raft_consensus.cc:2749] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260430 19:58:54.750555 5092 raft_consensus.cc:493] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.750718 5117 raft_consensus.cc:2889] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:54.751345 5117 raft_consensus.cc:2272] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:54.755707 4026 catalog_manager.cc:5671] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a7356e99e744525861e12c108cfa3fe (127.3.190.1). New cstate: current_term: 1 leader_uuid: "5a7356e99e744525861e12c108cfa3fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:54.759827 4514 tablet_service.cc:1511] Processing CreateTablet for tablet 418ec8b043624de6b760e1e56d469293 (DEFAULT_TABLE table=default.Table_3edd4511aea94058a799c4a3de95f968 [id=20b9b5e9b95e4d6596ebca2093bbefd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.760470 4196 tablet_service.cc:1511] Processing CreateTablet for tablet d272d84e3f3e4ff4ace102ac859426f6 (DEFAULT_TABLE table=default.Table_0c0f3cc776e847188b388e9ba7cb1285 [id=3b5da9bc474a44f896fce6cefddba90d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.761544 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d272d84e3f3e4ff4ace102ac859426f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.758212 4026 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:54.773829 4729 tablet_bootstrap.cc:492] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:54.774338 4729 ts_tablet_manager.cc:1403] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.262s user 0.016s sys 0.001s
I20260430 19:58:54.751108 5092 raft_consensus.cc:515] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.752573 4818 tablet_bootstrap.cc:492] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:54.758874 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet ea907f9bd93041de976a2f0c3b517dab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:54.764457 5174 consensus_queue.cc:1048] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:54.775393 4818 ts_tablet_manager.cc:1403] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.168s user 0.010s sys 0.000s
I20260430 19:58:54.777832 4818 raft_consensus.cc:359] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.777379 4189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7ea59485f046919b87e9fcaf8686ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.782510 4512 tablet_service.cc:1511] Processing CreateTablet for tablet 70e4d5af75d04f6d9091d2b086e4a6ba (DEFAULT_TABLE table=default.Table_74850f27abe54175909dc1d9c494804b [id=7ad67f36177f4ef18a33fa7a65cbef6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.785001 4506 tablet_service.cc:1511] Processing CreateTablet for tablet d0c2c08fac7248a1b3b52e2b43041dff (DEFAULT_TABLE table=default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c [id=d4bfa57fdfc64108ba1e909d0230fb51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.778786 5092 leader_election.cc:290] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:54.778561 4818 raft_consensus.cc:385] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.786953 4818 raft_consensus.cc:740] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:54.787596 4818 consensus_queue.cc:260] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:54.791903 4320 leader_election.cc:343] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: e4e87fda401047cfa7070ae95b1701a9
I20260430 19:58:54.792979 4511 tablet_service.cc:1511] Processing CreateTablet for tablet d846bbe4af0544e88ea5012ccc815881 (DEFAULT_TABLE table=default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.799963 4193 tablet_service.cc:1511] Processing CreateTablet for tablet 1fd70a5179dc4436bb2ba5a04f54f819 (DEFAULT_TABLE table=default.Table_4ee70552d7224ff9a90e149ed7a17de4 [id=735a21065fcf4d8e9952b3716f72a43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.801108 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd70a5179dc4436bb2ba5a04f54f819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.809540 4360 tablet_service.cc:1511] Processing CreateTablet for tablet 3ea24c4cdb4d43aba5762f0073488535 (DEFAULT_TABLE table=default.Table_4472ad7bad434d159c5275dc870c286c [id=51d4b8ce92574939bdc73ecd7c32d080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.811226 4510 tablet_service.cc:1511] Processing CreateTablet for tablet 6f0489b4631346f4952ef648e0653242 (DEFAULT_TABLE table=default.Table_60d3a33c8bfc4641ac60f0ea151c1225 [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.812849 4504 tablet_service.cc:1511] Processing CreateTablet for tablet ea907f9bd93041de976a2f0c3b517dab (DEFAULT_TABLE table=default.Table_1ef1c0b4ac9f4d5ca9433f755ecf83fe [id=766ef3562cd3467c9b6a8fce85828692]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.810721 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 7cb31c7051734e79966684015e05e631 (DEFAULT_TABLE table=default.Table_968941fa27b84dfab105dbb83df734ab [id=aa554293152a489cb6f3ad7fb3f67d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.813228 4354 tablet_service.cc:1511] Processing CreateTablet for tablet fb145e6d80f64bdb99965a3cf1e889f7 (DEFAULT_TABLE table=default.Table_a9a78f62539447b98602897b63dd48f8 [id=8b29d1f3dd5d4b45bcd3051cd989c8ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.811487 4352 tablet_service.cc:1511] Processing CreateTablet for tablet 418ec8b043624de6b760e1e56d469293 (DEFAULT_TABLE table=default.Table_3edd4511aea94058a799c4a3de95f968 [id=20b9b5e9b95e4d6596ebca2093bbefd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.812301 4358 tablet_service.cc:1511] Processing CreateTablet for tablet 5d405ddf1037400f9d87dda4c3b0e911 (DEFAULT_TABLE table=default.Table_a73bf5f11b0e46198f70577bc06dcf91 [id=ce2e2befe1a34e7eb0a693f48cf369c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.813350 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ea24c4cdb4d43aba5762f0073488535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.816457 4729 raft_consensus.cc:359] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.817175 4729 raft_consensus.cc:385] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:54.817229 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418ec8b043624de6b760e1e56d469293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.817492 4729 raft_consensus.cc:740] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:54.818743 4354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb145e6d80f64bdb99965a3cf1e889f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.818455 4192 tablet_service.cc:1511] Processing CreateTablet for tablet 33ebbb7e29724e8191209283085059f5 (DEFAULT_TABLE table=default.Table_a2d74c305d5f4d8ca3db561150380152 [id=a329d4d27b4f4266a37db9e021a05fbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.819536 4192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33ebbb7e29724e8191209283085059f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.820945 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cb31c7051734e79966684015e05e631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.780865 5117 ts_tablet_manager.cc:1916] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:54.823112 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d405ddf1037400f9d87dda4c3b0e911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.824095 4818 ts_tablet_manager.cc:1434] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.048s user 0.002s sys 0.002s
I20260430 19:58:54.834403 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 4219d92a298146ff8181728e1a95a0b7 (DEFAULT_TABLE table=default.Table_149dd50933e148538c23f32cf19ca2dd [id=fe0cd0ba31e44b6e9023e4cf663f42e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.835430 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4219d92a298146ff8181728e1a95a0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.836174 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 70e4d5af75d04f6d9091d2b086e4a6ba (DEFAULT_TABLE table=default.Table_74850f27abe54175909dc1d9c494804b [id=7ad67f36177f4ef18a33fa7a65cbef6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.837073 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e4d5af75d04f6d9091d2b086e4a6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.853624 4356 tablet_service.cc:1511] Processing CreateTablet for tablet d846bbe4af0544e88ea5012ccc815881 (DEFAULT_TABLE table=default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.854549 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d846bbe4af0544e88ea5012ccc815881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.854997 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798bb6007af849e99bf8e95d84624044" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:54.855566 4225 raft_consensus.cc:2468] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:54.863173 4818 tablet_bootstrap.cc:492] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:54.863262 4205 tablet_service.cc:1558] Processing DeleteTablet for tablet ea907f9bd93041de976a2f0c3b517dab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:54.864180 5167 raft_consensus.cc:493] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.864624 5167 raft_consensus.cc:515] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:54.867108 5167 leader_election.cc:290] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:54.868649 4818 tablet_bootstrap.cc:654] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:54.838997 4501 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa60e9fa2d844f8bad35b0a0cfdff93 (DEFAULT_TABLE table=default.Table_95badffa04c845919ea1d575c4df3244 [id=76bfbc7067e248d9828bfc8dd9b60a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.842144 4470 leader_election.cc:304] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:54.875046 4729 consensus_queue.cc:260] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.854727 4500 tablet_service.cc:1511] Processing CreateTablet for tablet ebd991541273429e9c2bdc018ee402f5 (DEFAULT_TABLE table=default.Table_cf9ef037d1634978a0ddbe0a06f32284 [id=567cc975cfeb451087657471b5b225d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.876147 5135 raft_consensus.cc:2749] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 19:58:54.885958 4024 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdf98515d8284e3b9b84480807e8108a consensus state has no leader
W20260430 19:58:54.894270 4161 leader_election.cc:343] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:54.903389 4317 leader_election.cc:343] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: e4e87fda401047cfa7070ae95b1701a9
I20260430 19:58:54.903827 4317 leader_election.cc:304] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:54.904860 5174 raft_consensus.cc:2749] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:54.909825 4499 tablet_service.cc:1511] Processing CreateTablet for tablet de1364857a1e40a5ab57166341222f11 (DEFAULT_TABLE table=default.Table_659324304ce3463fa7e7306b42d7ffb2 [id=399cf70cfcb74e08a56ee56480b0096c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.896934 4205 tablet_service.cc:1511] Processing CreateTablet for tablet fb145e6d80f64bdb99965a3cf1e889f7 (DEFAULT_TABLE table=default.Table_a9a78f62539447b98602897b63dd48f8 [id=8b29d1f3dd5d4b45bcd3051cd989c8ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.899122 4197 tablet_service.cc:1511] Processing CreateTablet for tablet 5d405ddf1037400f9d87dda4c3b0e911 (DEFAULT_TABLE table=default.Table_a73bf5f11b0e46198f70577bc06dcf91 [id=ce2e2befe1a34e7eb0a693f48cf369c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.900475 4200 tablet_service.cc:1511] Processing CreateTablet for tablet 3ea24c4cdb4d43aba5762f0073488535 (DEFAULT_TABLE table=default.Table_4472ad7bad434d159c5275dc870c286c [id=51d4b8ce92574939bdc73ecd7c32d080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.901597 4201 tablet_service.cc:1511] Processing CreateTablet for tablet 7cb31c7051734e79966684015e05e631 (DEFAULT_TABLE table=default.Table_968941fa27b84dfab105dbb83df734ab [id=aa554293152a489cb6f3ad7fb3f67d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.913472 4205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb145e6d80f64bdb99965a3cf1e889f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.913604 4498 tablet_service.cc:1511] Processing CreateTablet for tablet e2af1644a4d54fc8be95592c03afee51 (DEFAULT_TABLE table=default.Table_0172fa4b93034ecfb61f52915a6abb8f [id=22a5efdd2a9946d183a237eda1dadd8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.914089 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cb31c7051734e79966684015e05e631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.914902 4197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d405ddf1037400f9d87dda4c3b0e911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.915303 4022 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 6f0489b4631346f4952ef648e0653242
I20260430 19:58:54.917948 4497 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f0489b4631346f4952ef648e0653242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:54.918880 4200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ea24c4cdb4d43aba5762f0073488535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.924597 4021 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_f64798677d5443c394801041788a4e38" } new_table_name: "default.Table_a1a89c947b0d4f0597e563512b8a24d2" modify_external_catalogs: true
I20260430 19:58:54.927354 4024 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_6e4f222e6ba54644a2e2da32762001f1" } modify_external_catalogs: true
I20260430 19:58:54.927836 4024 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_6e4f222e6ba54644a2e2da32762001f1" } modify_external_catalogs: true
I20260430 19:58:54.929083 4025 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 798bb6007af849e99bf8e95d84624044 (table default.Table_425e3062d49141638642c50ca47fd102 [id=d963a034463343989c1d699cce2134a5]). Expected version 1 got 0
I20260430 19:58:54.935320 4742 ts_tablet_manager.cc:1929] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:54.935735 4742 log.cc:1199] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/0e62335e43c1410b8b3acdd9e2f47981
I20260430 19:58:54.936362 4742 ts_tablet_manager.cc:1950] T 0e62335e43c1410b8b3acdd9e2f47981 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
W20260430 19:58:54.936549 4470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 60986.6us
I20260430 19:58:54.937178 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 160) took 1657 ms. Trace:
I20260430 19:58:54.939100 4436 heartbeater.cc:344] Connected to a master server at 127.3.190.62:33781
I20260430 19:58:54.939602 4742 rpcz_store.cc:276] 0430 19:58:53.280061 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.280170 (+ 109us) service_pool.cc:225] Handling call
0430 19:58:54.501942 (+1221772us) ts_tablet_manager.cc:1151] Deleting tablet 0e62335e43c1410b8b3acdd9e2f47981
0430 19:58:54.501999 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:54.755223 (+253224us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.935199 (+179976us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:54.937126 (+ 1927us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1221278}
I20260430 19:58:54.941520 4361 tablet_service.cc:1511] Processing CreateTablet for tablet ebd991541273429e9c2bdc018ee402f5 (DEFAULT_TABLE table=default.Table_cf9ef037d1634978a0ddbe0a06f32284 [id=567cc975cfeb451087657471b5b225d3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:54.942066 4317 consensus_peers.cc:597] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Couldn't send request to peer 8a11a9a7f244435b8e84554e3fabbd0f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eafc058f392a45de9d30b521ebde0950. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:54.942521 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebd991541273429e9c2bdc018ee402f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.943174 4352 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa60e9fa2d844f8bad35b0a0cfdff93 (DEFAULT_TABLE table=default.Table_95badffa04c845919ea1d575c4df3244 [id=76bfbc7067e248d9828bfc8dd9b60a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.944088 4352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa60e9fa2d844f8bad35b0a0cfdff93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.954913 4729 ts_tablet_manager.cc:1434] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.180s user 0.007s sys 0.000s
I20260430 19:58:54.955987 4363 tablet_service.cc:1511] Processing CreateTablet for tablet ea907f9bd93041de976a2f0c3b517dab (DEFAULT_TABLE table=default.Table_1ef1c0b4ac9f4d5ca9433f755ecf83fe [id=766ef3562cd3467c9b6a8fce85828692]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.962229 5135 raft_consensus.cc:493] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.963299 5135 raft_consensus.cc:515] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.957353 5092 raft_consensus.cc:493] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:54.965171 5092 raft_consensus.cc:515] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:54.965647 5135 leader_election.cc:290] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:54.967031 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e5cb3aaa4044eb9dd06227882e2b42" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:54.967422 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea907f9bd93041de976a2f0c3b517dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.967945 4381 raft_consensus.cc:2468] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:54.967038 5092 leader_election.cc:290] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
W20260430 19:58:54.969832 4317 leader_election.cc:343] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 09f272340cd942e9b2af9e38a2bdec82
I20260430 19:58:54.970118 4470 leader_election.cc:304] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:54.970916 4497 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2e67993931481a8405bbe52823ec6c (DEFAULT_TABLE table=default.Table_1a0b8c0b26af42519951ec338ca684f3 [id=e09fedf32d084c48801ef71dbac47292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.970921 5135 raft_consensus.cc:2804] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260430 19:58:54.971741 4320 leader_election.cc:343] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: 09f272340cd942e9b2af9e38a2bdec82
I20260430 19:58:54.971946 5135 raft_consensus.cc:493] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:54.972098 4320 leader_election.cc:304] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:54.972289 5135 raft_consensus.cc:3060] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:54.973265 5092 raft_consensus.cc:2749] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:54.983050 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 6f0489b4631346f4952ef648e0653242 (DEFAULT_TABLE table=default.Table_60d3a33c8bfc4641ac60f0ea151c1225 [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.985121 4358 tablet_service.cc:1511] Processing CreateTablet for tablet de1364857a1e40a5ab57166341222f11 (DEFAULT_TABLE table=default.Table_659324304ce3463fa7e7306b42d7ffb2 [id=399cf70cfcb74e08a56ee56480b0096c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.988380 4358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de1364857a1e40a5ab57166341222f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.990159 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f0489b4631346f4952ef648e0653242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.985616 4357 tablet_service.cc:1511] Processing CreateTablet for tablet d0c2c08fac7248a1b3b52e2b43041dff (DEFAULT_TABLE table=default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c [id=d4bfa57fdfc64108ba1e909d0230fb51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.987215 4360 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f0489b4631346f4952ef648e0653242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:54.999895 4013 catalog_manager.cc:5075] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): alter failed for tablet fa59394fc34f419ca22a695b8dff4613 (table default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074]),no further retry: Not found: Tablet not found: fa59394fc34f419ca22a695b8dff4613
I20260430 19:58:54.998150 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c2c08fac7248a1b3b52e2b43041dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.004098 4818 tablet_bootstrap.cc:492] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:55.004513 4818 ts_tablet_manager.cc:1403] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.142s user 0.008s sys 0.000s
W20260430 19:58:54.938638 4161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 44406.7us
I20260430 19:58:54.988994 4199 tablet_service.cc:1511] Processing CreateTablet for tablet d0c2c08fac7248a1b3b52e2b43041dff (DEFAULT_TABLE table=default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c [id=d4bfa57fdfc64108ba1e909d0230fb51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.007069 4199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c2c08fac7248a1b3b52e2b43041dff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:55.007488 4161 leader_election.cc:343] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.007867 4161 leader_election.cc:304] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:54.990460 4204 tablet_service.cc:1511] Processing CreateTablet for tablet d846bbe4af0544e88ea5012ccc815881 (DEFAULT_TABLE table=default.Table_47de4e0e38f64d868f55414d6a9a4f6f [id=91e641e6e87f45ffb0a050142b221074]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:55.010655 4161 leader_election.cc:343] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.011291 4204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d846bbe4af0544e88ea5012ccc815881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.011420 5167 raft_consensus.cc:2749] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:55.012753 4360 tablet_service.cc:1511] Processing CreateTablet for tablet e2af1644a4d54fc8be95592c03afee51 (DEFAULT_TABLE table=default.Table_0172fa4b93034ecfb61f52915a6abb8f [id=22a5efdd2a9946d183a237eda1dadd8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.012872 4470 leader_election.cc:304] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:55.013706 4360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2af1644a4d54fc8be95592c03afee51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.014112 5192 raft_consensus.cc:2804] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:55.014439 5192 raft_consensus.cc:493] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:54.990769 4202 tablet_service.cc:1511] Processing CreateTablet for tablet 6f0489b4631346f4952ef648e0653242 (DEFAULT_TABLE table=default.Table_60d3a33c8bfc4641ac60f0ea151c1225 [id=37e26a1d9e4f40cc8fb14b9934ba3780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.014760 5192 raft_consensus.cc:3060] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.015503 4202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f0489b4631346f4952ef648e0653242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.015848 5135 raft_consensus.cc:515] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.001012 4200 tablet_service.cc:1511] Processing CreateTablet for tablet 418ec8b043624de6b760e1e56d469293 (DEFAULT_TABLE table=default.Table_3edd4511aea94058a799c4a3de95f968 [id=20b9b5e9b95e4d6596ebca2093bbefd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.992230 4194 tablet_service.cc:1511] Processing CreateTablet for tablet ea907f9bd93041de976a2f0c3b517dab (DEFAULT_TABLE table=default.Table_1ef1c0b4ac9f4d5ca9433f755ecf83fe [id=766ef3562cd3467c9b6a8fce85828692]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.019483 4194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea907f9bd93041de976a2f0c3b517dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:54.994262 4195 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa60e9fa2d844f8bad35b0a0cfdff93 (DEFAULT_TABLE table=default.Table_95badffa04c845919ea1d575c4df3244 [id=76bfbc7067e248d9828bfc8dd9b60a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.995820 4188 tablet_service.cc:1511] Processing CreateTablet for tablet ebd991541273429e9c2bdc018ee402f5 (DEFAULT_TABLE table=default.Table_cf9ef037d1634978a0ddbe0a06f32284 [id=567cc975cfeb451087657471b5b225d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:54.997526 4187 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f0489b4631346f4952ef648e0653242 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:55.019820 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:55.020593 5167 raft_consensus.cc:493] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:55.020992 5167 raft_consensus.cc:515] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.021824 4200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418ec8b043624de6b760e1e56d469293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.022711 5167 leader_election.cc:290] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:55.025033 4188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebd991541273429e9c2bdc018ee402f5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:55.021261 4161 leader_election.cc:343] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.025767 5192 raft_consensus.cc:515] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.026397 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e35e0981e649f684caa96499040c15" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:55.027580 4161 leader_election.cc:343] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.028541 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798bb6007af849e99bf8e95d84624044" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:55.028816 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2e67993931481a8405bbe52823ec6c (DEFAULT_TABLE table=default.Table_1a0b8c0b26af42519951ec338ca684f3 [id=e09fedf32d084c48801ef71dbac47292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.029073 4381 raft_consensus.cc:3060] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.029824 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2e67993931481a8405bbe52823ec6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.031507 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ea24c4cdb4d43aba5762f0073488535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.031405 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e5cb3aaa4044eb9dd06227882e2b42" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:55.031949 4383 raft_consensus.cc:3060] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.035523 5135 leader_election.cc:290] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:55.037250 4502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cb31c7051734e79966684015e05e631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.043038 4514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 418ec8b043624de6b760e1e56d469293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.047216 4512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e4d5af75d04f6d9091d2b086e4a6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.047945 4195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa60e9fa2d844f8bad35b0a0cfdff93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.051146 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c2c08fac7248a1b3b52e2b43041dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.051712 4511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d846bbe4af0544e88ea5012ccc815881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.055138 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f0489b4631346f4952ef648e0653242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.059248 4504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea907f9bd93041de976a2f0c3b517dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.059396 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e5cb3aaa4044eb9dd06227882e2b42" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:55.060395 4220 raft_consensus.cc:3060] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.063154 4501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa60e9fa2d844f8bad35b0a0cfdff93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.067276 4500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebd991541273429e9c2bdc018ee402f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.069674 4499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de1364857a1e40a5ab57166341222f11. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:55.074774 4013 catalog_manager.cc:4969] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 6f0489b4631346f4952ef648e0653242 because the tablet was not found. No further retry: Not found: Tablet not found: 6f0489b4631346f4952ef648e0653242
I20260430 19:58:55.075857 4818 raft_consensus.cc:359] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.076495 4818 raft_consensus.cc:385] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.076747 4818 raft_consensus.cc:740] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.087563 4383 raft_consensus.cc:2468] T 67e5cb3aaa4044eb9dd06227882e2b42 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:55.088662 4470 leader_election.cc:304] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:55.094450 4023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_46c35b095e404ff580d326f16651b1a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:55.095976 4023 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46c35b095e404ff580d326f16651b1a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.097676 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_33d79062c8a6462ca2522c82cf2826b5" } new_table_name: "default.Table_ab953a6fe3604a41aedf2c76252826b0" modify_external_catalogs: true
I20260430 19:58:55.086364 4220 raft_consensus.cc:2468] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:55.104379 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_efddd98fead448ea9245be8d1f3ccbbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:55.105798 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efddd98fead448ea9245be8d1f3ccbbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.104837 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e5cb3aaa4044eb9dd06227882e2b42" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:55.107014 4225 raft_consensus.cc:2376] T 67e5cb3aaa4044eb9dd06227882e2b42 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1. Re-sending same reply.
I20260430 19:58:55.108979 4498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2af1644a4d54fc8be95592c03afee51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.110013 4729 tablet_bootstrap.cc:492] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:55.112102 4381 raft_consensus.cc:2468] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:55.114537 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 180b5364dacd4ec785cf4fb365b0b046 (DEFAULT_TABLE table=default.Table_ba1d91aaad6e4d3db7fdae67076dd6c3 [id=08aac0066c4341d7b567cd7ccc8f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.115521 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180b5364dacd4ec785cf4fb365b0b046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.120174 4470 leader_election.cc:304] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:55.119517 5191 raft_consensus.cc:2804] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:55.121116 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_90a4740b66574861a91bc789cb56b353" } modify_external_catalogs: true
I20260430 19:58:55.121393 4497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2e67993931481a8405bbe52823ec6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.121624 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_90a4740b66574861a91bc789cb56b353" } modify_external_catalogs: true
I20260430 19:58:55.121726 5135 raft_consensus.cc:2804] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:55.125164 5135 raft_consensus.cc:697] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:55.125890 5135 consensus_queue.cc:237] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.126283 4029 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_2d7bda6327c844a2940c048b9bf2ff56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:55.127804 4029 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d7bda6327c844a2940c048b9bf2ff56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.129629 4742 tablet_replica.cc:333] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:55.130329 4509 tablet_service.cc:1511] Processing CreateTablet for tablet 180b5364dacd4ec785cf4fb365b0b046 (DEFAULT_TABLE table=default.Table_ba1d91aaad6e4d3db7fdae67076dd6c3 [id=08aac0066c4341d7b567cd7ccc8f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.131317 4742 raft_consensus.cc:2243] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:55.131453 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180b5364dacd4ec785cf4fb365b0b046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.133684 5192 leader_election.cc:290] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:55.160041 4742 raft_consensus.cc:2272] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:55.161798 4729 tablet_bootstrap.cc:654] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.174933 4742 ts_tablet_manager.cc:1916] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:55.174916 4470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 54801.3us
I20260430 19:58:55.188674 5137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc6c3c322c24063ad0cf6b243cf4755. Latest consensus state: current_term: 4 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:55.189481 5137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:55.190784 5175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } } }
I20260430 19:58:55.191450 5175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:58:55.192314 5143 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:55.192898 5143 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:55.214717 5138 raft_consensus.cc:493] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:55.215159 5138 raft_consensus.cc:515] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.246759 4818 consensus_queue.cc:260] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.250001 4818 ts_tablet_manager.cc:1434] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.245s user 0.007s sys 0.001s
I20260430 19:58:55.249363 5143 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:55.250882 4818 tablet_bootstrap.cc:492] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:55.253620 5117 ts_tablet_manager.cc:1929] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:55.254040 5117 log.cc:1199] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/6220177b3e3e4d18bd8e20e6d9d96be7
I20260430 19:58:55.254819 5117 ts_tablet_manager.cc:1950] T 6220177b3e3e4d18bd8e20e6d9d96be7 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:55.271875 4026 master_service.cc:954] ReplaceTablet: received request to replace tablet 4758b3d846434b2b8466033c083b7339 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:55.281113 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 160) took 1958 ms. Trace:
I20260430 19:58:55.281461 5117 rpcz_store.cc:276] 0430 19:58:53.322839 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.322996 (+ 157us) service_pool.cc:225] Handling call
0430 19:58:54.670177 (+1347181us) ts_tablet_manager.cc:1151] Deleting tablet 6220177b3e3e4d18bd8e20e6d9d96be7
0430 19:58:54.670228 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:54.971872 (+301644us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:55.253473 (+281601us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:55.281048 (+ 27575us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1346634}
I20260430 19:58:55.285904 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 6220177b3e3e4d18bd8e20e6d9d96be7 (table default.Table_0a03c960230b4206944b333b146c39c7 [id=26c468440dce47839c5c48e7bc76c173]) successfully deleted
I20260430 19:58:55.287408 5117 tablet_replica.cc:333] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:55.288592 5117 raft_consensus.cc:2243] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:55.295729 5117 raft_consensus.cc:2272] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:55.000993 4776 tablet_bootstrap.cc:654] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.178315 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798bb6007af849e99bf8e95d84624044" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:55.311214 4225 raft_consensus.cc:3060] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.311779 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 0e62335e43c1410b8b3acdd9e2f47981 (table default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620]) successfully deleted
I20260430 19:58:55.295210 4818 tablet_bootstrap.cc:654] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.318121 4225 raft_consensus.cc:2468] T 798bb6007af849e99bf8e95d84624044 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:55.324600 4999 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40712 (request call id 0) took 2783 ms. Trace:
I20260430 19:58:55.255939 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 70e4d5af75d04f6d9091d2b086e4a6ba (DEFAULT_TABLE table=default.Table_74850f27abe54175909dc1d9c494804b [id=7ad67f36177f4ef18a33fa7a65cbef6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.279958 4201 tablet_service.cc:1511] Processing CreateTablet for tablet de1364857a1e40a5ab57166341222f11 (DEFAULT_TABLE table=default.Table_659324304ce3463fa7e7306b42d7ffb2 [id=399cf70cfcb74e08a56ee56480b0096c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.327224 5117 ts_tablet_manager.cc:1916] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:55.336338 4729 tablet_bootstrap.cc:492] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:55.336504 5138 leader_election.cc:290] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:55.337706 4729 ts_tablet_manager.cc:1403] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.228s user 0.019s sys 0.016s
I20260430 19:58:55.338662 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:55.339982 4729 raft_consensus.cc:359] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:55.340359 4470 leader_election.cc:343] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.340566 4729 raft_consensus.cc:385] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.340794 4729 raft_consensus.cc:740] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.341321 4729 consensus_queue.cc:260] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.342941 4729 ts_tablet_manager.cc:1434] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260430 19:58:55.355146 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:55.358157 4729 tablet_bootstrap.cc:492] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:55.373986 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e4d5af75d04f6d9091d2b086e4a6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.374150 5201 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260430 19:58:55.376426 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:55.376981 4225 raft_consensus.cc:2468] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:55.378059 4470 leader_election.cc:304] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:55.393101 5138 raft_consensus.cc:2804] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:55.393456 5138 raft_consensus.cc:493] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:55.393772 5138 raft_consensus.cc:3060] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.407557 4985 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" instance_seqno: 1777579116008822) as {username='slave'} at 127.3.190.1:41501; Asking this server to re-register.
I20260430 19:58:55.424880 4729 tablet_bootstrap.cc:654] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.435987 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de1364857a1e40a5ab57166341222f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.439855 4274 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:55.409991 5138 raft_consensus.cc:515] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.451375 4024 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 84b02b29777341d3934f6a77967d79b0
I20260430 19:58:55.452316 5138 leader_election.cc:290] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
W20260430 19:58:55.452922 4021 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 619fb8dd24244d1ea81953668659df8a has no consensus state
I20260430 19:58:55.453047 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:55.453547 4225 raft_consensus.cc:3060] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:55.454186 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 608d010d0acc44b3bc06e0ccf9674bc0 has no consensus state
I20260430 19:58:55.456394 4381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
W20260430 19:58:55.469389 4470 leader_election.cc:343] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.481019 4776 tablet_bootstrap.cc:492] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:55.481480 4776 ts_tablet_manager.cc:1403] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.768s user 0.006s sys 0.005s
I20260430 19:58:55.482702 5191 raft_consensus.cc:697] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:55.483670 5191 consensus_queue.cc:237] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.498982 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet 84b02b29777341d3934f6a77967d79b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:55.503490 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 84b02b29777341d3934f6a77967d79b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84b02b29777341d3934f6a77967d79b0 already in progress: creating tablet
I20260430 19:58:55.498561 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 84b02b29777341d3934f6a77967d79b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:55.503221 4818 tablet_bootstrap.cc:492] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:55.509306 4818 ts_tablet_manager.cc:1403] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.259s user 0.025s sys 0.008s
I20260430 19:58:55.511780 4818 raft_consensus.cc:359] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:55.512508 4818 raft_consensus.cc:385] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.512753 4818 raft_consensus.cc:740] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.520824 3957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:55.533154 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:55.543812 4953 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:55.544803 5143 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:58:55.545534 4983 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" instance_seqno: 1777579118224413) as {username='slave'} at 127.3.190.2:49631; Asking this server to re-register.
I20260430 19:58:55.545768 4985 ts_manager.cc:194] Registered new tserver with Master: 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:55.550470 4436 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:55.554836 4225 raft_consensus.cc:2468] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:55.558698 4984 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" instance_seqno: 1777579120646175) as {username='slave'} at 127.3.190.3:45787; Asking this server to re-register.
I20260430 19:58:55.560271 5212 tablet.cc:1724] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:55.562882 5166 consensus_queue.cc:1048] T 745f04d04091443ca1c39949d005dd69 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:58:55.565871 4189 consensus_meta.cc:287] T ec7ea59485f046919b87e9fcaf8686ca P 5a7356e99e744525861e12c108cfa3fe: Time spent flushing consensus metadata: real 0.584s user 0.008s sys 0.000s
I20260430 19:58:55.555903 4470 leader_election.cc:304] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:55.576184 4776 raft_consensus.cc:359] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.576792 4776 raft_consensus.cc:385] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.577050 4776 raft_consensus.cc:740] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.577594 4581 heartbeater.cc:461] Registering TS with master...
I20260430 19:58:55.580613 4984 ts_manager.cc:194] Registered new tserver with Master: d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:55.580647 5197 raft_consensus.cc:2804] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:55.581103 5197 raft_consensus.cc:697] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:55.590659 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 3d36c2538ec84e0889c4e53975a2ff1c
I20260430 19:58:55.593770 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d36c2538ec84e0889c4e53975a2ff1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:55.581785 5197 consensus_queue.cc:237] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.595146 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d36c2538ec84e0889c4e53975a2ff1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:55.596747 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 3d36c2538ec84e0889c4e53975a2ff1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d36c2538ec84e0889c4e53975a2ff1c already in progress: creating tablet
I20260430 19:58:55.598021 4020 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3 [id=2c86f02b8cd14685913a92ee25eb0483] alter complete (version 1)
I20260430 19:58:55.513609 4818 consensus_queue.cc:260] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:55.603366 4818 ts_tablet_manager.cc:1434] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.094s user 0.006s sys 0.000s
I20260430 19:58:55.606545 4818 tablet_bootstrap.cc:492] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:55.631047 4776 consensus_queue.cc:260] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.633173 4776 ts_tablet_manager.cc:1434] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.151s user 0.006s sys 0.001s
I20260430 19:58:55.634897 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_6b7e49e3e1ee4ce4bfa8075c27cba432"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:55.636610 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b7e49e3e1ee4ce4bfa8075c27cba432 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.637789 4818 tablet_bootstrap.cc:654] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.645498 4023 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:55.646304 5117 ts_tablet_manager.cc:1929] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:55.650414 5135 raft_consensus.cc:493] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:55.651117 5135 raft_consensus.cc:515] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.654383 5135 leader_election.cc:290] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:55.658057 4029 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_e298a596df714d4494a339954849530b" } new_table_name: "default.Table_e6c53456a3fb43c28d96db9f42b8f560" modify_external_catalogs: true
I20260430 19:58:55.659536 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d410f3b3b04e649b79dda04c0891ee" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:55.659821 4028 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_3e3a927708124dbfa583240f38fdbe55" } new_table_name: "default.Table_79c910b252344461a992f1d1dc1bd625" modify_external_catalogs: true
I20260430 19:58:55.660106 4225 raft_consensus.cc:2468] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:55.661274 4470 leader_election.cc:304] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:55.667743 5198 raft_consensus.cc:2804] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:55.668171 5198 raft_consensus.cc:493] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:55.668704 5198 raft_consensus.cc:3060] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.679006 4023 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_15c9bde5c7f54d888d9c298975bd5e06" } modify_external_catalogs: true
I20260430 19:58:55.682282 4023 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_15c9bde5c7f54d888d9c298975bd5e06" } modify_external_catalogs: true
I20260430 19:58:55.684619 4776 tablet_bootstrap.cc:492] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:55.690299 4729 tablet_bootstrap.cc:492] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:55.690783 4729 ts_tablet_manager.cc:1403] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.333s user 0.013s sys 0.006s
W20260430 19:58:55.692741 4015 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 619fb8dd24244d1ea81953668659df8a (table default.Table_a1a89c947b0d4f0597e563512b8a24d2 [id=a3be52c2582f4074839ab189b1461c86], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 619fb8dd24244d1ea81953668659df8a has no consensus state
W20260430 19:58:55.697147 4028 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c2adbc9008d4397ade546b1b3f59f13 has no consensus state
I20260430 19:58:55.707845 4729 raft_consensus.cc:359] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.708475 4729 raft_consensus.cc:385] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.708725 4729 raft_consensus.cc:740] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.709352 4729 consensus_queue.cc:260] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.712563 5187 tablet.cc:1724] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260430 19:58:55.731787 4158 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 174562us
W20260430 19:58:55.732466 4158 leader_election.cc:343] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Not found: Tablet not found: 51e35e0981e649f684caa96499040c15
I20260430 19:58:55.732863 4158 leader_election.cc:304] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
W20260430 19:58:55.733445 4158 raft_consensus.cc:2686] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260430 19:58:55.734097 4158 leader_election.cc:343] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.734464 4158 leader_election.cc:304] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:55.735167 5166 raft_consensus.cc:2749] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:55.737295 4201 tablet_service.cc:1511] Processing CreateTablet for tablet e2af1644a4d54fc8be95592c03afee51 (DEFAULT_TABLE table=default.Table_0172fa4b93034ecfb61f52915a6abb8f [id=22a5efdd2a9946d183a237eda1dadd8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.738355 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2af1644a4d54fc8be95592c03afee51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.745548 4198 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2e67993931481a8405bbe52823ec6c (DEFAULT_TABLE table=default.Table_1a0b8c0b26af42519951ec338ca684f3 [id=e09fedf32d084c48801ef71dbac47292]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:55.746716 4198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2e67993931481a8405bbe52823ec6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:55.669600 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d410f3b3b04e649b79dda04c0891ee" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:55.767599 5117 log.cc:1199] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/f3db0439f4f042a4a9cb38ebd3764ab5
I20260430 19:58:55.768327 5117 ts_tablet_manager.cc:1950] T f3db0439f4f042a4a9cb38ebd3764ab5 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
W20260430 19:58:55.769768 4470 leader_election.cc:343] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.770370 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 161) took 2447 ms. Trace:
I20260430 19:58:55.771095 5117 rpcz_store.cc:276] 0430 19:58:53.323385 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.323604 (+ 219us) service_pool.cc:225] Handling call
0430 19:58:55.287211 (+1963607us) ts_tablet_manager.cc:1151] Deleting tablet f3db0439f4f042a4a9cb38ebd3764ab5
0430 19:58:55.287255 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:55.592239 (+304984us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:55.646181 (+ 53942us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:55.770316 (+124135us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1962647}
I20260430 19:58:55.765764 5229 tablet.cc:1724] T 0832f9191b7f4c3683f37cce8966b266 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:55.772634 4729 ts_tablet_manager.cc:1434] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.082s user 0.005s sys 0.001s
I20260430 19:58:55.776250 4729 tablet_bootstrap.cc:492] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:55.780344 4189 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d36c2538ec84e0889c4e53975a2ff1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:55.781440 4189 tablet_service.cc:1558] Processing DeleteTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:55.782829 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 84b02b29777341d3934f6a77967d79b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:55.729995 5198 raft_consensus.cc:515] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.788205 4776 tablet_bootstrap.cc:654] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.793911 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d410f3b3b04e649b79dda04c0891ee" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:55.794461 4225 raft_consensus.cc:3060] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.798629 4029 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cf1378d9973a48079c8e2b0bea84dc2a" } modify_external_catalogs: true
I20260430 19:58:55.799125 4029 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_cf1378d9973a48079c8e2b0bea84dc2a" } modify_external_catalogs: true
I20260430 19:58:55.800699 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_bf80dfa8947243c69b46cbac49bfe41a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:55.801528 4225 raft_consensus.cc:2468] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
W20260430 19:58:55.802152 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf80dfa8947243c69b46cbac49bfe41a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.802526 4470 leader_election.cc:304] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:55.803398 5191 raft_consensus.cc:2804] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:55.803810 5191 raft_consensus.cc:697] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:55.804548 5191 consensus_queue.cc:237] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.805401 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_3b4274cea2394743828eb6d1a31fdbbe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:55.806892 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b4274cea2394743828eb6d1a31fdbbe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.807786 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_c507e6b6dbb2496e90f0ec6eab3a02c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:55.809157 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c507e6b6dbb2496e90f0ec6eab3a02c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.809978 5198 leader_election.cc:290] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:55.821054 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet 5563cfa73f194db69a6978d348f1c073 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:55.815320 5225 tablet.cc:1724] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:55.798924 4984 ts_manager.cc:194] Registered new tserver with Master: 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:55.835621 5212 tablet.cc:1724] T d4e34f3b1bf84f72855cf87683c5e318 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:55.856284 4729 tablet_bootstrap.cc:654] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.859848 4742 ts_tablet_manager.cc:1929] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:55.860298 4742 log.cc:1199] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/51e35e0981e649f684caa96499040c15
I20260430 19:58:55.860934 4742 ts_tablet_manager.cc:1950] T 51e35e0981e649f684caa96499040c15 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:55.873108 4729 tablet_bootstrap.cc:492] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:55.874861 4729 ts_tablet_manager.cc:1403] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.099s user 0.010s sys 0.000s
I20260430 19:58:55.884593 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_7e85838feea147dca5c3b2e154d654b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:55.885375 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet f3db0439f4f042a4a9cb38ebd3764ab5 (table default.Table_7e8d384dfa62442fbf585d952b02fe29 [id=977b7bf53cdf45158ea1fb1a4d2fdf05]) successfully deleted
W20260430 19:58:55.886085 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e85838feea147dca5c3b2e154d654b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:55.886507 5117 tablet_replica.cc:333] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:55.887259 5117 raft_consensus.cc:2243] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:55.887667 5117 raft_consensus.cc:2272] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:55.758466 4818 tablet_bootstrap.cc:492] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:55.788897 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d410f3b3b04e649b79dda04c0891ee" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:55.889151 4818 ts_tablet_manager.cc:1403] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.283s user 0.034s sys 0.021s
W20260430 19:58:55.889961 4470 leader_election.cc:343] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.749464 5228 tablet.cc:1724] T 0832f9191b7f4c3683f37cce8966b266 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:55.843987 5232 tablet.cc:1724] T d4e34f3b1bf84f72855cf87683c5e318 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:55.891479 4818 raft_consensus.cc:359] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.892063 4818 raft_consensus.cc:385] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.891769 4729 raft_consensus.cc:359] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:55.892289 4818 raft_consensus.cc:740] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.892411 4729 raft_consensus.cc:385] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.892642 4729 raft_consensus.cc:740] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.892827 4818 consensus_queue.cc:260] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.893251 4729 consensus_queue.cc:260] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:55.895103 4729 ts_tablet_manager.cc:1434] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260430 19:58:55.895891 4729 tablet_bootstrap.cc:492] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:55.906944 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:55.910152 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_a74cec79d3954c01ba9ebd8fef4cb8f3" } new_table_name: "default.Table_2b693cd1b0344bb4957052067f59f377" modify_external_catalogs: true
I20260430 19:58:55.914729 4818 ts_tablet_manager.cc:1434] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260430 19:58:55.915717 4818 tablet_bootstrap.cc:492] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:55.917982 4729 tablet_bootstrap.cc:654] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.925653 4025 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_3a24adb795934ac7a893366f60e21964" } new_table_name: "default.Table_dff1a0b8ecb24c2b839430012ba75f23" modify_external_catalogs: true
I20260430 19:58:55.949528 4818 tablet_bootstrap.cc:654] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:55.975392 5138 raft_consensus.cc:493] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:55.975932 4818 tablet_bootstrap.cc:492] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:55.976356 4818 ts_tablet_manager.cc:1403] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.007s
I20260430 19:58:55.975910 5138 raft_consensus.cc:515] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:55.977913 5138 leader_election.cc:290] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:55.978886 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:55.978788 4818 raft_consensus.cc:359] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.979089 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:55.979467 4220 raft_consensus.cc:2468] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
W20260430 19:58:55.980515 4470 leader_election.cc:343] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:55.981145 4470 leader_election.cc:304] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:55.982151 4818 raft_consensus.cc:385] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:55.985531 5138 raft_consensus.cc:2804] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:55.985893 5138 raft_consensus.cc:493] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:55.985944 4818 raft_consensus.cc:740] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:55.986202 5138 raft_consensus.cc:3060] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:55.986727 4818 consensus_queue.cc:260] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:55.994117 4818 ts_tablet_manager.cc:1434] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260430 19:58:55.997681 4818 tablet_bootstrap.cc:492] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
W20260430 19:58:56.001520 4025 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdf98515d8284e3b9b84480807e8108a consensus state has no leader
I20260430 19:58:56.006073 5092 raft_consensus.cc:493] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.008584 5092 raft_consensus.cc:515] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.013188 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395f97fdf3834cb9aaabd261e1bf76c5" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:56.013821 4220 raft_consensus.cc:2468] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:56.014981 4320 leader_election.cc:304] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:56.015717 5174 raft_consensus.cc:2804] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:56.016045 5174 raft_consensus.cc:493] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:56.016336 5174 raft_consensus.cc:3060] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.010462 5092 leader_election.cc:290] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.028028 4818 tablet_bootstrap.cc:654] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.037424 5174 raft_consensus.cc:515] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.038715 5092 raft_consensus.cc:493] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.039129 5092 raft_consensus.cc:515] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.040364 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_aa816c09c98b4870b953e6da12736c04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:56.042066 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa816c09c98b4870b953e6da12736c04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.042806 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:56.044138 4317 leader_election.cc:343] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:56.045398 5092 leader_election.cc:290] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.046685 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:56.047248 4220 raft_consensus.cc:2468] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:56.048465 4320 leader_election.cc:304] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:56.051914 5092 raft_consensus.cc:2804] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:56.052037 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_a86a2cb6fb394ec19c8ef6599fdfff22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:56.053033 5092 raft_consensus.cc:493] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:56.053339 5092 raft_consensus.cc:3060] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:56.053512 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a86a2cb6fb394ec19c8ef6599fdfff22 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.055088 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_839a70940e6f4a6a96ac5cec16e26a91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:56.056502 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_839a70940e6f4a6a96ac5cec16e26a91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.075042 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395f97fdf3834cb9aaabd261e1bf76c5" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:56.075567 4220 raft_consensus.cc:3060] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.076316 4818 tablet_bootstrap.cc:492] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.077780 4818 ts_tablet_manager.cc:1403] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.080s user 0.006s sys 0.004s
I20260430 19:58:56.080104 4818 raft_consensus.cc:359] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.080772 4818 raft_consensus.cc:385] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.081060 4818 raft_consensus.cc:740] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.081657 4818 consensus_queue.cc:260] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.083841 4818 ts_tablet_manager.cc:1434] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260430 19:58:56.087797 4818 tablet_bootstrap.cc:492] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.093458 4818 tablet_bootstrap.cc:654] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.105424 4818 tablet_bootstrap.cc:492] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.105938 4818 ts_tablet_manager.cc:1403] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.012s
I20260430 19:58:56.108295 4818 raft_consensus.cc:359] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.109136 4818 raft_consensus.cc:385] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.109424 4818 raft_consensus.cc:740] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.110055 4818 consensus_queue.cc:260] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.112111 4818 ts_tablet_manager.cc:1434] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260430 19:58:56.122270 4818 tablet_bootstrap.cc:492] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.111071 5174 leader_election.cc:290] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.123611 5174 raft_consensus.cc:493] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.124094 5174 raft_consensus.cc:515] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.126749 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:56.128906 5174 leader_election.cc:290] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:56.093743 4220 raft_consensus.cc:2468] T 395f97fdf3834cb9aaabd261e1bf76c5 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:56.133414 4320 leader_election.cc:304] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:56.134042 5242 raft_consensus.cc:493] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.134279 4818 tablet_bootstrap.cc:654] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.134560 5242 raft_consensus.cc:515] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.136909 5242 leader_election.cc:290] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.137506 5242 raft_consensus.cc:2804] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:56.138228 5242 raft_consensus.cc:697] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:56.139091 5242 consensus_queue.cc:237] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.141760 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:56.142452 4220 raft_consensus.cc:2393] T ed157967cdad480f88963a59fe85b54b P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8345ec7f57c4399a8a4afe4dd43439e in current term 1: Already voted for candidate 8a11a9a7f244435b8e84554e3fabbd0f in this term.
I20260430 19:58:56.146939 5228 tablet.cc:1724] T 6cef0197c00b4ab7a11e2e5123a4c277 P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:56.148732 5092 raft_consensus.cc:515] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.151504 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:56.152110 4225 raft_consensus.cc:3060] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.152962 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_df7f2cb342be4e7aaab6f206bb9e3c0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:56.154430 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df7f2cb342be4e7aaab6f206bb9e3c0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.161839 4225 raft_consensus.cc:2468] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:56.163033 4320 leader_election.cc:304] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:56.164062 5243 raft_consensus.cc:2804] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:56.164479 5243 raft_consensus.cc:697] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:56.165431 5243 consensus_queue.cc:237] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.183297 4022 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_e6c53456a3fb43c28d96db9f42b8f560 [id=9713902a096f4953a4287b4078bea519] alter complete (version 2)
I20260430 19:58:56.205138 4020 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_9d21910e9cb14cc6a75a5432b143c915" } new_table_name: "default.Table_f4523dfb8ab9414bad907abc56460874" modify_external_catalogs: true
I20260430 19:58:55.998488 5138 raft_consensus.cc:515] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.212036 4534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed157967cdad480f88963a59fe85b54b" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:56.213199 5092 leader_election.cc:290] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.214002 4026 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_dff1a0b8ecb24c2b839430012ba75f23" } new_table_name: "default.Table_d2376323c69a4567887757a6e5f84d59" modify_external_catalogs: true
I20260430 19:58:56.214489 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:56.215061 4528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:56.216267 5225 tablet.cc:1724] T 6cef0197c00b4ab7a11e2e5123a4c277 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:56.213305 4317 leader_election.cc:304] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:56.211565 4533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4764dc427b4a48309de914c4f5e655fe" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:56.217906 4470 leader_election.cc:343] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:56.218247 5092 raft_consensus.cc:3060] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.218389 4533 raft_consensus.cc:2393] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8345ec7f57c4399a8a4afe4dd43439e in current term 1: Already voted for candidate 5a7356e99e744525861e12c108cfa3fe in this term.
I20260430 19:58:56.218343 5138 leader_election.cc:290] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
W20260430 19:58:56.222167 4317 leader_election.cc:343] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:56.223235 4317 leader_election.cc:304] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:56.224020 5243 raft_consensus.cc:3060] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.230331 4776 tablet_bootstrap.cc:492] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:56.230940 4776 ts_tablet_manager.cc:1403] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.546s user 0.007s sys 0.004s
I20260430 19:58:56.231966 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395f97fdf3834cb9aaabd261e1bf76c5" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:56.232545 4530 raft_consensus.cc:3060] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.233495 4776 raft_consensus.cc:359] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.234087 4776 raft_consensus.cc:385] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.234325 4776 raft_consensus.cc:740] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.242002 5166 consensus_queue.cc:1048] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:56.244724 4818 tablet_bootstrap.cc:492] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.245177 4818 ts_tablet_manager.cc:1403] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.123s user 0.017s sys 0.024s
I20260430 19:58:56.219141 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:56.246358 4225 raft_consensus.cc:3060] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.246446 5117 ts_tablet_manager.cc:1916] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:56.248044 5243 raft_consensus.cc:2749] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:56.251029 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 162) took 2760 ms. Trace:
I20260430 19:58:56.251235 5166 consensus_queue.cc:1048] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:56.251456 4742 rpcz_store.cc:276] 0430 19:58:53.491081 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.491232 (+ 151us) service_pool.cc:225] Handling call
0430 19:58:55.129427 (+1638195us) ts_tablet_manager.cc:1151] Deleting tablet 51e35e0981e649f684caa96499040c15
0430 19:58:55.129512 (+ 85us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:55.621432 (+491920us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:55.859708 (+238276us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:56.250946 (+391238us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1555047}
I20260430 19:58:56.253142 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 163) took 2723 ms. Trace:
I20260430 19:58:56.253791 5212 tablet.cc:1724] T 6cef0197c00b4ab7a11e2e5123a4c277 P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:56.264281 4818 raft_consensus.cc:359] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.268455 4818 raft_consensus.cc:385] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.268491 4198 tablet_service.cc:1511] Processing CreateTablet for tablet 180b5364dacd4ec785cf4fb365b0b046 (DEFAULT_TABLE table=default.Table_ba1d91aaad6e4d3db7fdae67076dd6c3 [id=08aac0066c4341d7b567cd7ccc8f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.268981 4818 raft_consensus.cc:740] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.270519 4530 raft_consensus.cc:2468] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:56.270495 4818 consensus_queue.cc:260] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.276182 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395f97fdf3834cb9aaabd261e1bf76c5" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:56.276350 5135 consensus_queue.cc:1048] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:56.283939 5135 consensus_queue.cc:1048] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:56.292284 5092 raft_consensus.cc:2749] T ed157967cdad480f88963a59fe85b54b P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:56.293617 4818 ts_tablet_manager.cc:1434] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.048s user 0.009s sys 0.000s
I20260430 19:58:56.294464 4818 tablet_bootstrap.cc:492] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.294492 4532 raft_consensus.cc:2376] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1. Re-sending same reply.
I20260430 19:58:56.298468 4729 tablet_bootstrap.cc:492] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:56.313813 4818 tablet_bootstrap.cc:654] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.328732 4729 ts_tablet_manager.cc:1403] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.433s user 0.015s sys 0.005s
I20260430 19:58:56.331779 4729 raft_consensus.cc:359] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.344480 4818 tablet_bootstrap.cc:492] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.344945 4818 ts_tablet_manager.cc:1403] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.051s user 0.010s sys 0.002s
I20260430 19:58:56.347615 4818 raft_consensus.cc:359] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.348310 4818 raft_consensus.cc:385] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.348559 4818 raft_consensus.cc:740] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.349298 4818 consensus_queue.cc:260] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.351468 4818 ts_tablet_manager.cc:1434] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260430 19:58:56.352329 4818 tablet_bootstrap.cc:492] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.253490 4742 rpcz_store.cc:276] 0430 19:58:53.529688 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.529799 (+ 111us) service_pool.cc:225] Handling call
0430 19:58:56.252738 (+2722939us) ts_tablet_manager.cc:1151] Deleting tablet f50116c63b0747099de2d234ca3faaac
0430 19:58:56.252800 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:56.253094 (+ 294us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2710591}
I20260430 19:58:56.362186 4742 tablet_replica.cc:333] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:56.362922 4742 raft_consensus.cc:2243] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:56.363426 4742 raft_consensus.cc:2272] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:56.363838 4818 tablet_bootstrap.cc:654] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.364960 4742 ts_tablet_manager.cc:1916] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:56.291301 4776 consensus_queue.cc:260] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.371472 4025 master_service.cc:954] ReplaceTablet: received request to replace tablet ebd991541273429e9c2bdc018ee402f5 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:56.378162 4198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180b5364dacd4ec785cf4fb365b0b046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.379081 4776 ts_tablet_manager.cc:1434] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.148s user 0.008s sys 0.000s
I20260430 19:58:56.383741 5250 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:56.385437 5250 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:56.386255 5117 ts_tablet_manager.cc:1929] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:56.388154 3958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:56.390347 5117 log.cc:1199] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/51e35e0981e649f684caa96499040c15
I20260430 19:58:56.390540 5250 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:56.391245 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:56.392658 5117 ts_tablet_manager.cc:1950] T 51e35e0981e649f684caa96499040c15 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:56.393481 4953 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:56.394344 4818 tablet_bootstrap.cc:492] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.394788 4818 ts_tablet_manager.cc:1403] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.007s
I20260430 19:58:56.397338 4776 tablet_bootstrap.cc:492] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:56.398149 4818 raft_consensus.cc:359] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.399930 4818 raft_consensus.cc:385] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.400103 4818 raft_consensus.cc:740] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.400658 4818 consensus_queue.cc:260] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.400925 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 51e35e0981e649f684caa96499040c15 (table default.Table_4f04fa211ec3447488b05b4f5b157487 [id=56ead7419d654e5dad1631c264ec87a5]) successfully deleted
W20260430 19:58:56.401654 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet f50116c63b0747099de2d234ca3faaac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f50116c63b0747099de2d234ca3faaac already in progress: creating tablet
I20260430 19:58:56.403434 4776 tablet_bootstrap.cc:654] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.396368 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 163) took 2906 ms. Trace:
I20260430 19:58:56.416829 4776 tablet_bootstrap.cc:492] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:56.417196 5117 rpcz_store.cc:276] 0430 19:58:53.490053 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.490154 (+ 101us) service_pool.cc:225] Handling call
0430 19:58:55.886347 (+2396193us) ts_tablet_manager.cc:1151] Deleting tablet 51e35e0981e649f684caa96499040c15
0430 19:58:55.886392 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:56.326395 (+440003us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:56.386138 (+ 59743us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:56.396311 (+ 10173us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2395527}
I20260430 19:58:56.419248 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 51e35e0981e649f684caa96499040c15 (table default.Table_4f04fa211ec3447488b05b4f5b157487 [id=56ead7419d654e5dad1631c264ec87a5]) successfully deleted
I20260430 19:58:56.420473 4024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_9e269db71b714e8084c34773f70acf2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:56.421602 4818 ts_tablet_manager.cc:1434] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
W20260430 19:58:56.421921 4024 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e269db71b714e8084c34773f70acf2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.422502 4818 tablet_bootstrap.cc:492] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.425033 4776 ts_tablet_manager.cc:1403] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.000s
I20260430 19:58:56.427096 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 164) took 2935 ms. Trace:
I20260430 19:58:56.436259 5117 rpcz_store.cc:276] 0430 19:58:53.491810 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.498732 (+ 6922us) service_pool.cc:225] Handling call
0430 19:58:56.425484 (+2926752us) ts_tablet_manager.cc:1151] Deleting tablet f50116c63b0747099de2d234ca3faaac
0430 19:58:56.425526 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:56.425745 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2878265}
W20260430 19:58:56.439539 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet f50116c63b0747099de2d234ca3faaac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f50116c63b0747099de2d234ca3faaac already in progress: creating tablet
I20260430 19:58:56.440721 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 173) took 2673 ms. Trace:
I20260430 19:58:56.441079 5117 rpcz_store.cc:276] 0430 19:58:53.766980 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.767114 (+ 134us) service_pool.cc:225] Handling call
0430 19:58:56.440405 (+2673291us) ts_tablet_manager.cc:1151] Deleting tablet 4758b3d846434b2b8466033c083b7339
0430 19:58:56.440445 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:56.440668 (+ 223us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2569518}
W20260430 19:58:56.442512 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 4758b3d846434b2b8466033c083b7339 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4758b3d846434b2b8466033c083b7339 already in progress: creating tablet
I20260430 19:58:56.438370 4776 raft_consensus.cc:359] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:56.444025 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 174) took 2675 ms. Trace:
I20260430 19:58:56.444376 4776 raft_consensus.cc:385] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.444679 4776 raft_consensus.cc:740] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.445019 5117 rpcz_store.cc:276] 0430 19:58:53.768389 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.768520 (+ 131us) service_pool.cc:225] Handling call
0430 19:58:56.443731 (+2675211us) ts_tablet_manager.cc:1151] Deleting tablet f5c54c37bdae458d88594a3f5c401a48
0430 19:58:56.443769 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:56.443979 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2572606}
W20260430 19:58:56.447613 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet f5c54c37bdae458d88594a3f5c401a48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5c54c37bdae458d88594a3f5c401a48 already in progress: creating tablet
I20260430 19:58:56.449707 4022 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_e0a6f83819644441aa6fbbeb7868eefa [id=d92b629d07c647f6abf34593325a5a16] alter complete (version 1)
W20260430 19:58:56.449963 4026 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdf98515d8284e3b9b84480807e8108a consensus state has no leader
W20260430 19:58:56.450855 4020 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3452fa52c20142f7bf396bc182a829a7 consensus state has no leader
I20260430 19:58:56.446017 4776 consensus_queue.cc:260] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:56.463512 5117 tablet_replica.cc:333] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:56.464346 4729 raft_consensus.cc:385] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.466267 4729 raft_consensus.cc:740] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.470909 4225 raft_consensus.cc:2468] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:56.477829 4818 tablet_bootstrap.cc:654] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.482354 5250 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:58:56.468179 4729 consensus_queue.cc:260] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.485047 4729 ts_tablet_manager.cc:1434] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.155s user 0.005s sys 0.000s
I20260430 19:58:56.485189 5201 catalog_manager.cc:1269] Loaded cluster ID: 56a2bf65e645471990682d6d15252142
I20260430 19:58:56.486229 5201 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: loading cluster ID for follower catalog manager: success
I20260430 19:58:56.508416 5117 raft_consensus.cc:2243] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:56.513662 4028 catalog_manager.cc:5671] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:56.516127 4357 tablet_service.cc:1511] Processing CreateTablet for tablet d39b081ba75a44e7832e39d06776fb46 (DEFAULT_TABLE table=default.Table_88431e83ce7e468aa602238e637bfa58 [id=e6b21ff4b54346719582c248c03ea0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.519061 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39b081ba75a44e7832e39d06776fb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.520619 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_2428915980164d0f81ac0081377f4d8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:56.522025 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2428915980164d0f81ac0081377f4d8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.521837 4022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_d35337f4d43a45e48e14b9f523681d04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:56.523203 4022 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d35337f4d43a45e48e14b9f523681d04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.527226 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_5ff3da24c8fe4a829ccbedd744bfb835"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:56.528697 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ff3da24c8fe4a829ccbedd744bfb835 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.530261 4818 tablet_bootstrap.cc:492] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.530682 4818 ts_tablet_manager.cc:1403] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.108s user 0.010s sys 0.000s
I20260430 19:58:56.532909 4818 raft_consensus.cc:359] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.533479 4818 raft_consensus.cc:385] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.533705 4818 raft_consensus.cc:740] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.534879 5117 pending_rounds.cc:70] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:56.542778 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 8e87f3808161419bbf08cd64afef9f0f (DEFAULT_TABLE table=default.Table_16a0661d7f5f41c18b86be9745bc54a7 [id=999555379df8409f95632c045b622be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.545105 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e87f3808161419bbf08cd64afef9f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.546834 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 619fb8dd24244d1ea81953668659df8a (DEFAULT_TABLE table=default.Table_a1a89c947b0d4f0597e563512b8a24d2 [id=a3be52c2582f4074839ab189b1461c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.547758 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 619fb8dd24244d1ea81953668659df8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.552559 5092 raft_consensus.cc:493] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.553011 5092 raft_consensus.cc:515] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.553572 4020 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:56.555317 5092 leader_election.cc:290] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:56.558779 5117 pending_rounds.cc:77] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964141742997504 op_type: NO_OP noop_request { }
I20260430 19:58:56.559238 5117 raft_consensus.cc:2889] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:56.561604 5117 raft_consensus.cc:2272] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:56.563809 5201 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: acquiring CA information for follower catalog manager: success
I20260430 19:58:56.563817 5117 ts_tablet_manager.cc:1916] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:56.571260 4189 tablet_service.cc:1511] Processing CreateTablet for tablet 619fb8dd24244d1ea81953668659df8a (DEFAULT_TABLE table=default.Table_a1a89c947b0d4f0597e563512b8a24d2 [id=a3be52c2582f4074839ab189b1461c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.572317 4189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 619fb8dd24244d1ea81953668659df8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.577791 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1bb19388d94a1e939793a9ddffc384 (DEFAULT_TABLE table=default.Table_7e4d0c964a2545d3b6ee8e7ea7e77ea0 [id=5bbca2de72764621935ca3402e558813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.587296 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1bb19388d94a1e939793a9ddffc384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.510747 4509 tablet_service.cc:1511] Processing CreateTablet for tablet d39b081ba75a44e7832e39d06776fb46 (DEFAULT_TABLE table=default.Table_88431e83ce7e468aa602238e637bfa58 [id=e6b21ff4b54346719582c248c03ea0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.590430 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39b081ba75a44e7832e39d06776fb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.590862 4818 consensus_queue.cc:260] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.593676 4729 tablet_bootstrap.cc:492] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:56.520548 5138 raft_consensus.cc:493] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.595032 5138 raft_consensus.cc:515] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.596894 5138 leader_election.cc:290] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.541252 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 8e87f3808161419bbf08cd64afef9f0f (DEFAULT_TABLE table=default.Table_16a0661d7f5f41c18b86be9745bc54a7 [id=999555379df8409f95632c045b622be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.597849 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e87f3808161419bbf08cd64afef9f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.554368 4020 catalog_manager.cc:5671] T 798bb6007af849e99bf8e95d84624044 P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } }
I20260430 19:58:56.599613 4020 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 798bb6007af849e99bf8e95d84624044 (table default.Table_425e3062d49141638642c50ca47fd102 [id=d963a034463343989c1d699cce2134a5]). Expected version 1 got 0
I20260430 19:58:56.548493 4510 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1bb19388d94a1e939793a9ddffc384 (DEFAULT_TABLE table=default.Table_7e4d0c964a2545d3b6ee8e7ea7e77ea0 [id=5bbca2de72764621935ca3402e558813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.604058 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1bb19388d94a1e939793a9ddffc384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.549603 4498 tablet_service.cc:1511] Processing CreateTablet for tablet 619fb8dd24244d1ea81953668659df8a (DEFAULT_TABLE table=default.Table_a1a89c947b0d4f0597e563512b8a24d2 [id=a3be52c2582f4074839ab189b1461c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.607496 4498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 619fb8dd24244d1ea81953668659df8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.606541 4818 ts_tablet_manager.cc:1434] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.076s user 0.005s sys 0.000s
I20260430 19:58:56.612103 4193 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1bb19388d94a1e939793a9ddffc384 (DEFAULT_TABLE table=default.Table_7e4d0c964a2545d3b6ee8e7ea7e77ea0 [id=5bbca2de72764621935ca3402e558813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.613066 4193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1bb19388d94a1e939793a9ddffc384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.616488 4504 tablet_service.cc:1511] Processing CreateTablet for tablet 96eee0fe5edb42558df6edb5659eb65d (DEFAULT_TABLE table=default.Table_d55e71cfbdcd48b280a28b690222278a [id=a7f6dbf74c14400a9060489a6094fb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.617565 4504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96eee0fe5edb42558df6edb5659eb65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.619946 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 96eee0fe5edb42558df6edb5659eb65d (DEFAULT_TABLE table=default.Table_d55e71cfbdcd48b280a28b690222278a [id=a7f6dbf74c14400a9060489a6094fb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.620954 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96eee0fe5edb42558df6edb5659eb65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.558944 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3423379d1e3d4020a4241bf8917f1050" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:56.567533 4470 leader_election.cc:304] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters: d8345ec7f57c4399a8a4afe4dd43439e
W20260430 19:58:56.622314 4317 leader_election.cc:343] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:56.622886 4470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 55427.9us
I20260430 19:58:56.623289 4818 tablet_bootstrap.cc:492] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.624764 5191 raft_consensus.cc:2804] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:56.625185 5191 raft_consensus.cc:697] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:56.625952 5191 consensus_queue.cc:237] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.628070 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
W20260430 19:58:56.630911 4470 leader_election.cc:343] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:56.632058 5198 consensus_queue.cc:1048] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:56.632736 5135 consensus_queue.cc:1048] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:58:56.634405 4470 consensus_peers.cc:597] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:56.638231 5198 consensus_queue.cc:1048] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:56.641877 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0efaecfef44457a5128e878796c061" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:56.642578 4225 raft_consensus.cc:2393] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:56.643435 4470 leader_election.cc:304] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:56.644320 5198 raft_consensus.cc:3060] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.650831 4818 tablet_bootstrap.cc:654] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.655684 5198 raft_consensus.cc:2749] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:56.655995 4027 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_4e505a72d8d140029f5450a3d9572d4b" } new_table_name: "default.Table_e76d755fd286415aa45c478c973a6ddb" modify_external_catalogs: true
I20260430 19:58:56.497947 4742 ts_tablet_manager.cc:1929] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:56.518126 4776 ts_tablet_manager.cc:1434] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.091s user 0.005s sys 0.000s
I20260430 19:58:56.540436 4201 tablet_service.cc:1511] Processing CreateTablet for tablet d39b081ba75a44e7832e39d06776fb46 (DEFAULT_TABLE table=default.Table_88431e83ce7e468aa602238e637bfa58 [id=e6b21ff4b54346719582c248c03ea0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.668658 4742 log.cc:1199] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/70d410f3b3b04e649b79dda04c0891ee
I20260430 19:58:56.669562 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39b081ba75a44e7832e39d06776fb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.669639 4742 ts_tablet_manager.cc:1950] T 70d410f3b3b04e649b79dda04c0891ee P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:56.544421 4203 tablet_service.cc:1511] Processing CreateTablet for tablet 8e87f3808161419bbf08cd64afef9f0f (DEFAULT_TABLE table=default.Table_16a0661d7f5f41c18b86be9745bc54a7 [id=999555379df8409f95632c045b622be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.557873 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3423379d1e3d4020a4241bf8917f1050" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:56.675199 4196 tablet_service.cc:1511] Processing CreateTablet for tablet 96eee0fe5edb42558df6edb5659eb65d (DEFAULT_TABLE table=default.Table_d55e71cfbdcd48b280a28b690222278a [id=a7f6dbf74c14400a9060489a6094fb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.675477 4220 raft_consensus.cc:2468] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:56.675231 4203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e87f3808161419bbf08cd64afef9f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.676661 4320 leader_election.cc:304] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:56.676870 4818 tablet_bootstrap.cc:492] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.677299 4818 ts_tablet_manager.cc:1403] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.010s
I20260430 19:58:56.677630 5092 raft_consensus.cc:2804] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:56.677985 5092 raft_consensus.cc:493] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:56.678273 5092 raft_consensus.cc:3060] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.678869 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96eee0fe5edb42558df6edb5659eb65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.679755 4818 raft_consensus.cc:359] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:56.680403 4818 raft_consensus.cc:385] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.681602 4818 raft_consensus.cc:740] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.683077 4818 consensus_queue.cc:260] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:56.685536 4776 tablet_bootstrap.cc:492] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:56.692090 5092 raft_consensus.cc:515] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.700742 4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3423379d1e3d4020a4241bf8917f1050" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:56.701277 4220 raft_consensus.cc:3060] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:56.696419 4818 ts_tablet_manager.cc:1434] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260430 19:58:56.703384 4818 tablet_bootstrap.cc:492] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.708635 5247 raft_consensus.cc:493] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.708969 5201 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:58:56.709075 5247 raft_consensus.cc:515] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:56.711050 5247 leader_election.cc:290] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.713979 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70d410f3b3b04e649b79dda04c0891ee" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:56.714303 4818 tablet_bootstrap.cc:654] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.707811 4220 raft_consensus.cc:2468] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:56.718312 4320 leader_election.cc:304] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:56.719172 5247 raft_consensus.cc:2804] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:56.719589 5247 raft_consensus.cc:697] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:56.720319 5247 consensus_queue.cc:237] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.731216 4818 tablet_bootstrap.cc:492] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
W20260430 19:58:56.731398 4016 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 608d010d0acc44b3bc06e0ccf9674bc0 (table default.Table_ab953a6fe3604a41aedf2c76252826b0 [id=ebde464aa40e44618799a21c9a9e7076], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 608d010d0acc44b3bc06e0ccf9674bc0 consensus state has no leader
I20260430 19:58:56.731604 4818 ts_tablet_manager.cc:1403] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.001s
I20260430 19:58:56.733446 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 608d010d0acc44b3bc06e0ccf9674bc0 (DEFAULT_TABLE table=default.Table_ab953a6fe3604a41aedf2c76252826b0 [id=ebde464aa40e44618799a21c9a9e7076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.733906 4818 raft_consensus.cc:359] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:56.734465 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 608d010d0acc44b3bc06e0ccf9674bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.734496 4818 raft_consensus.cc:385] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.734758 4818 raft_consensus.cc:740] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.740255 4818 consensus_queue.cc:260] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:56.742094 4818 ts_tablet_manager.cc:1434] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260430 19:58:56.742908 4818 tablet_bootstrap.cc:492] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.747876 4818 tablet_bootstrap.cc:654] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.757329 4029 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 239a2c6a534f4f478e436d14f94cd762
W20260430 19:58:56.762238 4028 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.3:35079 (request call id 50) took 971 ms (client timeout 1000 ms). Trace:
W20260430 19:58:56.762554 4028 rpcz_store.cc:269] 0430 19:58:55.790365 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:55.791046 (+ 681us) service_pool.cc:225] Handling call
0430 19:58:56.762180 (+971134us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:56.524649 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:56.524985 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:56.525008 (+ 23us) write_op.cc:435] Acquired schema lock
0430 19:58:56.525020 (+ 12us) tablet.cc:662] Decoding operations
0430 19:58:56.525247 (+ 227us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:56.525278 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:56.525298 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:56.525419 (+ 121us) tablet.cc:701] Row locks acquired
0430 19:58:56.525431 (+ 12us) write_op.cc:260] PREPARE: finished
0430 19:58:56.525523 (+ 92us) write_op.cc:270] Start()
0430 19:58:56.525591 (+ 68us) write_op.cc:276] Timestamp: P: 1777579136525304 usec, L: 0
0430 19:58:56.525605 (+ 14us) op_driver.cc:348] REPLICATION: starting
0430 19:58:56.525999 (+ 394us) log.cc:844] Serialized 527 byte log entry
0430 19:58:56.751552 (+225553us) op_driver.cc:464] REPLICATION: finished
0430 19:58:56.759069 (+ 7517us) write_op.cc:301] APPLY: starting
0430 19:58:56.759115 (+ 46us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:56.759194 (+ 79us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:56.759265 (+ 71us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:56.759403 (+ 138us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:56.759515 (+ 112us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:56.759593 (+ 78us) write_op.cc:312] APPLY: finished
0430 19:58:56.759866 (+ 273us) log.cc:844] Serialized 23 byte log entry
0430 19:58:56.760064 (+ 198us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:56.760160 (+ 96us) write_op.cc:454] Released schema lock
0430 19:58:56.761153 (+ 993us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":7362,"num_ops":1,"prepare.queue_time_us":8751,"prepare.run_cpu_time_us":1516,"prepare.run_wall_time_us":1515,"replication_time_us":225726}]]}
I20260430 19:58:56.769294 4023 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 958cf4732e254c75bda36a6a550b93ef
I20260430 19:58:56.770668 4023 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:55130 (request call id 277) took 1113 ms. Trace:
I20260430 19:58:56.770953 4023 rpcz_store.cc:276] 0430 19:58:55.657379 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:55.678709 (+ 21330us) service_pool.cc:225] Handling call
0430 19:58:55.703617 (+ 24908us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0430 19:58:55.711511 (+ 7894us) catalog_manager.cc:2815] Modifying in-memory table state
0430 19:58:55.711652 (+ 141us) catalog_manager.cc:2824] Locking tablets
0430 19:58:56.507976 (+796324us) catalog_manager.cc:2838] Removing table and tablets from system table
0430 19:58:56.768947 (+260971us) catalog_manager.cc:2856] Removing table from by-name map
0430 19:58:56.769121 (+ 174us) catalog_manager.cc:2872] Committing in-memory state
0430 19:58:56.769211 (+ 90us) catalog_manager.cc:2876] Aborting table tasks
0430 19:58:56.770617 (+ 1406us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:56.509878 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:56.510228 (+ 350us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:56.510249 (+ 21us) write_op.cc:435] Acquired schema lock
0430 19:58:56.510262 (+ 13us) tablet.cc:662] Decoding operations
0430 19:58:56.510553 (+ 291us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:56.510607 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:56.510628 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0430 19:58:56.510888 (+ 260us) tablet.cc:701] Row locks acquired
0430 19:58:56.510902 (+ 14us) write_op.cc:260] PREPARE: finished
0430 19:58:56.511010 (+ 108us) write_op.cc:270] Start()
0430 19:58:56.511093 (+ 83us) write_op.cc:276] Timestamp: P: 1777579136510788 usec, L: 0
0430 19:58:56.511109 (+ 16us) op_driver.cc:348] REPLICATION: starting
0430 19:58:56.511570 (+ 461us) log.cc:844] Serialized 756 byte log entry
0430 19:58:56.750890 (+239320us) op_driver.cc:464] REPLICATION: finished
0430 19:58:56.752441 (+ 1551us) write_op.cc:301] APPLY: starting
0430 19:58:56.752551 (+ 110us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:56.752720 (+ 169us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:56.752849 (+ 129us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:56.753415 (+ 566us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:56.753715 (+ 300us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0430 19:58:56.753807 (+ 92us) write_op.cc:312] APPLY: finished
0430 19:58:56.754224 (+ 417us) log.cc:844] Serialized 29 byte log entry
0430 19:58:56.754453 (+ 229us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:56.754539 (+ 86us) write_op.cc:454] Released schema lock
0430 19:58:56.766619 (+ 12080us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:56.766978 (+ 359us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:56.767020 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1375,"apply.run_cpu_time_us":2509,"apply.run_wall_time_us":14507,"num_ops":2,"prepare.queue_time_us":159,"prepare.run_cpu_time_us":1886,"prepare.run_wall_time_us":1886,"replication_time_us":239688}]]}
I20260430 19:58:56.774691 4025 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet ebd991541273429e9c2bdc018ee402f5
I20260430 19:58:56.775949 4025 catalog_manager.cc:6741] ReplaceTablet: tablet ebd991541273429e9c2bdc018ee402f5 deleted and replaced by tablet faa69ffb02e445d3a06c95358d38a0e7
I20260430 19:58:56.778621 4497 tablet_service.cc:1558] Processing DeleteTablet for tablet 239a2c6a534f4f478e436d14f94cd762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:53008
W20260430 19:58:56.780361 4320 leader_election.cc:343] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Not found: Tablet not found: 70d410f3b3b04e649b79dda04c0891ee
I20260430 19:58:56.780472 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet 958cf4732e254c75bda36a6a550b93ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:56.780861 4320 leader_election.cc:304] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:56.781951 5247 raft_consensus.cc:2749] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:56.782258 4512 tablet_service.cc:1558] Processing DeleteTablet for tablet ebd991541273429e9c2bdc018ee402f5 with delete_type TABLET_DATA_DELETED (replaced by tablet faa69ffb02e445d3a06c95358d38a0e7) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:56.794207 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 958cf4732e254c75bda36a6a550b93ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:56.794124 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3423379d1e3d4020a4241bf8917f1050" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:56.795046 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet ebd991541273429e9c2bdc018ee402f5 with delete_type TABLET_DATA_DELETED (replaced by tablet faa69ffb02e445d3a06c95358d38a0e7) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:56.795867 4317 leader_election.cc:343] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:56.796597 4193 tablet_service.cc:1558] Processing DeleteTablet for tablet 958cf4732e254c75bda36a6a550b93ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:56.796944 4197 tablet_service.cc:1558] Processing DeleteTablet for tablet ebd991541273429e9c2bdc018ee402f5 with delete_type TABLET_DATA_DELETED (replaced by tablet faa69ffb02e445d3a06c95358d38a0e7) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:56.801695 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 239a2c6a534f4f478e436d14f94cd762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:56.804579 4776 tablet_bootstrap.cc:654] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:56.812072 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 239a2c6a534f4f478e436d14f94cd762 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 239a2c6a534f4f478e436d14f94cd762 already in progress: creating tablet
I20260430 19:58:56.817219 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 239a2c6a534f4f478e436d14f94cd762 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:55 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:56.821686 4029 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_a24540b6ec7948578c87d89d6116fdb9" } new_table_name: "default.Table_63f9f99010054a52adde13533fc41269" modify_external_catalogs: true
W20260430 19:58:56.825609 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 958cf4732e254c75bda36a6a550b93ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 958cf4732e254c75bda36a6a550b93ef already in progress: creating tablet
I20260430 19:58:56.711138 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 171) took 2610 ms. Trace:
W20260430 19:58:56.827708 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet ebd991541273429e9c2bdc018ee402f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebd991541273429e9c2bdc018ee402f5 already in progress: creating tablet
I20260430 19:58:56.827654 4028 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_79f86b4f9880405db20eade1a857c8e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:56.827939 4742 rpcz_store.cc:276] 0430 19:58:54.100302 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.211992 (+111690us) service_pool.cc:225] Handling call
0430 19:58:56.362013 (+2150021us) ts_tablet_manager.cc:1151] Deleting tablet 70d410f3b3b04e649b79dda04c0891ee
0430 19:58:56.362076 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:56.373942 (+ 11866us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:56.497824 (+123882us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:56.711053 (+213229us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2148770}
W20260430 19:58:56.829248 4028 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79f86b4f9880405db20eade1a857c8e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:56.829552 4742 tablet_replica.cc:333] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:56.835918 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 70d410f3b3b04e649b79dda04c0891ee (table default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]) successfully deleted
I20260430 19:58:56.851238 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 608d010d0acc44b3bc06e0ccf9674bc0 (DEFAULT_TABLE table=default.Table_ab953a6fe3604a41aedf2c76252826b0 [id=ebde464aa40e44618799a21c9a9e7076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.852231 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 608d010d0acc44b3bc06e0ccf9674bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.809388 5092 leader_election.cc:290] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:56.812793 4818 tablet_bootstrap.cc:492] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:56.862984 4818 ts_tablet_manager.cc:1403] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.120s user 0.009s sys 0.000s
I20260430 19:58:56.866353 4729 tablet_bootstrap.cc:654] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:56.866614 4818 raft_consensus.cc:359] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.867295 4818 raft_consensus.cc:385] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:56.867552 4818 raft_consensus.cc:740] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:56.869314 4818 consensus_queue.cc:260] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.871457 5247 raft_consensus.cc:493] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.872040 5247 raft_consensus.cc:515] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:56.874359 5247 leader_election.cc:290] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:56.877159 4818 ts_tablet_manager.cc:1434] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260430 19:58:56.880856 5117 ts_tablet_manager.cc:1929] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:56.889937 5117 log.cc:1199] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/db0efaecfef44457a5128e878796c061
I20260430 19:58:56.889946 4361 tablet_service.cc:1511] Processing CreateTablet for tablet fe3bee2131a5418bad503adcbf072fd8 (DEFAULT_TABLE table=default.Table_13e7445cfe4d4ed9ae643ac477a76530 [id=0082f7434428427aaf5f501e3c91e03b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.890858 5117 ts_tablet_manager.cc:1950] T db0efaecfef44457a5128e878796c061 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:56.890975 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe3bee2131a5418bad503adcbf072fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.893644 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 175) took 3060 ms. Trace:
I20260430 19:58:56.894018 5117 rpcz_store.cc:276] 0430 19:58:53.832772 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.833670 (+ 898us) service_pool.cc:225] Handling call
0430 19:58:56.452851 (+2619181us) ts_tablet_manager.cc:1151] Deleting tablet db0efaecfef44457a5128e878796c061
0430 19:58:56.453137 (+ 286us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:56.754151 (+301014us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:56.880743 (+126592us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:56.893224 (+ 12481us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2557764}
I20260430 19:58:56.895148 5117 tablet_replica.cc:333] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:56.895879 5117 raft_consensus.cc:2243] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:56.896333 5117 raft_consensus.cc:2272] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:56.896800 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet db0efaecfef44457a5128e878796c061 (table default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c [id=d4bfa57fdfc64108ba1e909d0230fb51]) successfully deleted
I20260430 19:58:56.899732 4818 tablet_bootstrap.cc:492] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:56.927321 4361 tablet_service.cc:1511] Processing CreateTablet for tablet 96901efdb27047c0a48040bff11be8b4 (DEFAULT_TABLE table=default.Table_3ca34fb096c24aa692126c78e7d66dbd [id=57190c6f3f2d4d1a95337651dffd89ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.928301 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96901efdb27047c0a48040bff11be8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.950484 4024 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 798bb6007af849e99bf8e95d84624044 (table default.Table_425e3062d49141638642c50ca47fd102 [id=d963a034463343989c1d699cce2134a5]). Expected version 1 got 0
I20260430 19:58:56.952039 4024 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
W20260430 19:58:56.956830 4029 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66e94750de27416e9adcf60fccf2ace9 consensus state has no leader
I20260430 19:58:56.961135 5247 raft_consensus.cc:493] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:56.961624 5247 raft_consensus.cc:515] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:56.963510 5247 leader_election.cc:290] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:56.963647 4028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_47de4e0e38f64d868f55414d6a9a4f6f" } modify_external_catalogs: true
I20260430 19:58:56.964581 4028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_47de4e0e38f64d868f55414d6a9a4f6f" } modify_external_catalogs: true
I20260430 19:58:56.968329 4529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:56.969731 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ba03500c0540cda44debe362278e47" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:56.971632 4317 leader_election.cc:343] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:56.977602 4029 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_9cfed595445342da8d6b800243a1a693" } modify_external_catalogs: true
I20260430 19:58:56.978104 4029 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_9cfed595445342da8d6b800243a1a693" } modify_external_catalogs: true
I20260430 19:58:56.981405 4509 tablet_service.cc:1511] Processing CreateTablet for tablet fe3bee2131a5418bad503adcbf072fd8 (DEFAULT_TABLE table=default.Table_13e7445cfe4d4ed9ae643ac477a76530 [id=0082f7434428427aaf5f501e3c91e03b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.982403 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe3bee2131a5418bad503adcbf072fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:56.983364 4513 tablet_service.cc:1511] Processing CreateTablet for tablet 96901efdb27047c0a48040bff11be8b4 (DEFAULT_TABLE table=default.Table_3ca34fb096c24aa692126c78e7d66dbd [id=57190c6f3f2d4d1a95337651dffd89ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:56.984262 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96901efdb27047c0a48040bff11be8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:57.005501 5247 raft_consensus.cc:493] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:57.006179 5247 raft_consensus.cc:515] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.009027 5247 leader_election.cc:290] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:57.012003 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:57.012043 4026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_c9c47d2286e64a108c57b78454f22006"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:57.013588 4026 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c9c47d2286e64a108c57b78454f22006 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:58:57.015139 4317 leader_election.cc:343] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:56.901705 5117 ts_tablet_manager.cc:1916] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:57.022838 4027 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 33d13905b1144012b0e12039e9b3a087 consensus state has no leader
I20260430 19:58:57.023638 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_d5e01da3c4624e18a5371fb5388caea4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:57.025152 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5e01da3c4624e18a5371fb5388caea4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:57.028182 4742 raft_consensus.cc:2243] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:57.028221 4027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_01b1e6993b4b46debd2be7d95a5b163d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:58:57.028716 4742 raft_consensus.cc:2272] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:57.029695 4027 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01b1e6993b4b46debd2be7d95a5b163d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:57.032562 4776 tablet_bootstrap.cc:492] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:57.033063 4776 ts_tablet_manager.cc:1403] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.348s user 0.002s sys 0.010s
I20260430 19:58:57.035344 4776 raft_consensus.cc:359] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.035943 4776 raft_consensus.cc:385] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.036185 4776 raft_consensus.cc:740] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.036707 4776 consensus_queue.cc:260] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.038853 4776 ts_tablet_manager.cc:1434] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260430 19:58:57.039659 4776 tablet_bootstrap.cc:492] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:57.044945 4818 tablet_bootstrap.cc:654] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:57.045967 4189 tablet_service.cc:1511] Processing CreateTablet for tablet 608d010d0acc44b3bc06e0ccf9674bc0 (DEFAULT_TABLE table=default.Table_ab953a6fe3604a41aedf2c76252826b0 [id=ebde464aa40e44618799a21c9a9e7076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:57.047516 4189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 608d010d0acc44b3bc06e0ccf9674bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:57.053072 4384 raft_consensus.cc:3060] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.073588 4742 ts_tablet_manager.cc:1916] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:57.078995 4022 master_service.cc:954] ReplaceTablet: received request to replace tablet 6f0489b4631346f4952ef648e0653242 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:57.088619 5117 ts_tablet_manager.cc:1929] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:57.089018 5117 log.cc:1199] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/0e62335e43c1410b8b3acdd9e2f47981
I20260430 19:58:57.089704 5117 ts_tablet_manager.cc:1950] T 0e62335e43c1410b8b3acdd9e2f47981 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:57.107134 4029 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet d703e0b6c6cb433d9c49b5d7956ae360
I20260430 19:58:57.113652 4510 tablet_service.cc:1558] Processing DeleteTablet for tablet d703e0b6c6cb433d9c49b5d7956ae360 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:56 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:57.115029 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:57.115612 4223 raft_consensus.cc:2468] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
W20260430 19:58:57.116174 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet d703e0b6c6cb433d9c49b5d7956ae360 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d703e0b6c6cb433d9c49b5d7956ae360 already in progress: creating tablet
I20260430 19:58:57.117245 4028 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet d846bbe4af0544e88ea5012ccc815881
W20260430 19:58:57.119454 4158 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 46373.6us
I20260430 19:58:57.119927 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 176) took 3176 ms. Trace:
I20260430 19:58:57.120275 5117 rpcz_store.cc:276] 0430 19:58:53.943819 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:53.943964 (+ 145us) service_pool.cc:225] Handling call
0430 19:58:56.895012 (+2951048us) ts_tablet_manager.cc:1151] Deleting tablet 0e62335e43c1410b8b3acdd9e2f47981
0430 19:58:56.895049 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:57.042408 (+147359us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:57.088499 (+ 46091us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:57.119852 (+ 31353us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2950495}
I20260430 19:58:57.121563 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 180) took 3035 ms. Trace:
I20260430 19:58:57.121876 5117 rpcz_store.cc:276] 0430 19:58:54.085858 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.086005 (+ 147us) service_pool.cc:225] Handling call
0430 19:58:57.121214 (+3035209us) ts_tablet_manager.cc:1151] Deleting tablet 34a6d662493e428aa475979abbffc9d7
0430 19:58:57.121258 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:57.121511 (+ 253us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3034553}
I20260430 19:58:57.122161 4196 tablet_service.cc:1511] Processing CreateTablet for tablet fe3bee2131a5418bad503adcbf072fd8 (DEFAULT_TABLE table=default.Table_13e7445cfe4d4ed9ae643ac477a76530 [id=0082f7434428427aaf5f501e3c91e03b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:57.122850 5117 tablet_replica.cc:333] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:57.122999 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "081a060c44924bcda09ffc8fe88012b3" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:57.123407 4196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe3bee2131a5418bad503adcbf072fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:57.123601 4225 raft_consensus.cc:2393] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8345ec7f57c4399a8a4afe4dd43439e in current term 1: Already voted for candidate 8a11a9a7f244435b8e84554e3fabbd0f in this term.
I20260430 19:58:57.123577 5117 raft_consensus.cc:2243] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:57.124007 5117 raft_consensus.cc:2272] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:57.138860 4510 tablet_service.cc:1558] Processing DeleteTablet for tablet d846bbe4af0544e88ea5012ccc815881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:56 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:57.138918 4320 leader_election.cc:304] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:57.140039 4320 leader_election.cc:304] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:57.140734 5174 raft_consensus.cc:2804] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:57.141213 5174 raft_consensus.cc:493] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:57.141671 5174 raft_consensus.cc:3060] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:57.147253 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet d846bbe4af0544e88ea5012ccc815881 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d846bbe4af0544e88ea5012ccc815881 already in progress: creating tablet
W20260430 19:58:57.147504 4198 consensus_meta.cc:287] T 180b5364dacd4ec785cf4fb365b0b046 P 5a7356e99e744525861e12c108cfa3fe: Time spent flushing consensus metadata: real 0.583s user 0.010s sys 0.000s
I20260430 19:58:57.148922 4198 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41996 (request call id 208) took 1369 ms. Trace:
I20260430 19:58:57.149421 4198 rpcz_store.cc:276] 0430 19:58:55.779423 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:56.267679 (+488256us) service_pool.cc:225] Handling call
0430 19:58:56.377716 (+110037us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0430 19:58:56.377800 (+ 84us) ts_tablet_manager.cc:682] Creating new metadata...
0430 19:58:56.564098 (+186298us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:57.148144 (+584046us) tablet_replica.cc:206] Creating consensus instance
0430 19:58:57.148843 (+ 699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260430 19:58:57.151508 4776 tablet_bootstrap.cc:654] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:57.151476 5247 raft_consensus.cc:3060] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.154749 5198 consensus_queue.cc:1048] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:58:57.165056 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_98fb1fb33cc84e1f8938cb0412c85545"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:57.166774 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98fb1fb33cc84e1f8938cb0412c85545 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:57.151777 5117 ts_tablet_manager.cc:1916] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:57.179143 4029 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_46c35b095e404ff580d326f16651b1a9" } new_table_name: "default.Table_2e10c9590ede4aca976afa6a1e397c7b" modify_external_catalogs: true
I20260430 19:58:57.185601 5092 consensus_queue.cc:1048] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:57.187942 5247 raft_consensus.cc:2749] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:57.190203 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ba03500c0540cda44debe362278e47" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:57.196980 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 0e62335e43c1410b8b3acdd9e2f47981 (table default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620]) successfully deleted
W20260430 19:58:57.197645 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 34a6d662493e428aa475979abbffc9d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34a6d662493e428aa475979abbffc9d7 already in progress: creating tablet
I20260430 19:58:57.208253 5145 mvcc.cc:204] Tried to move back new op lower bound from 7280964124966027264 to 7280964080824262656. Current Snapshot: MvccSnapshot[applied={T|T < 7280964124466057216 or (T in {7280964124717760512})}]
I20260430 19:58:57.225827 4742 ts_tablet_manager.cc:1929] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:57.230264 4510 tablet_service.cc:1511] Processing CreateTablet for tablet f9bf7670880746848933517216df3bf3 (DEFAULT_TABLE table=default.Table_cc310773560b43f89ee4742daa2360c6 [id=e40e56a9eaeb45d2be94d02eb6dbb5c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:57.233054 4022 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_8ebf726da3884df1b2b6805cb6b147a5" } new_table_name: "default.Table_84d9207cf43849148c90cd0b2fa6c17f" modify_external_catalogs: true
I20260430 19:58:57.251212 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9bf7670880746848933517216df3bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:57.197739 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet d703e0b6c6cb433d9c49b5d7956ae360 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:56 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:57.198758 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet d846bbe4af0544e88ea5012ccc815881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:56 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:57.212297 5174 raft_consensus.cc:515] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.268812 5174 leader_election.cc:290] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:57.309875 4025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_ae13850aa92e4dd89ee374d24cd28885"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:57.311447 4025 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae13850aa92e4dd89ee374d24cd28885 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:57.317083 4023 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_2428915980164d0f81ac0081377f4d8f" } modify_external_catalogs: true
I20260430 19:58:57.328749 4023 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_2428915980164d0f81ac0081377f4d8f" } modify_external_catalogs: true
I20260430 19:58:57.348249 4729 tablet_bootstrap.cc:492] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:57.348721 4729 ts_tablet_manager.cc:1403] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.760s user 0.012s sys 0.012s
I20260430 19:58:57.357963 4028 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 798bb6007af849e99bf8e95d84624044 (table default.Table_425e3062d49141638642c50ca47fd102 [id=d963a034463343989c1d699cce2134a5]). Expected version 1 got 0
I20260430 19:58:57.352802 4729 raft_consensus.cc:359] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:57.380879 4729 raft_consensus.cc:385] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.381167 4729 raft_consensus.cc:740] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.381811 4729 consensus_queue.cc:260] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:57.387913 4729 ts_tablet_manager.cc:1434] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20260430 19:58:57.388729 4729 tablet_bootstrap.cc:492] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:57.394003 4729 tablet_bootstrap.cc:654] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:57.226794 4158 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.3:37953, user_credentials={real_user=slave}} blocked reactor thread for 95962.4us
I20260430 19:58:57.226490 5255 raft_consensus.cc:493] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:57.398801 4196 tablet_service.cc:1558] Processing DeleteTablet for tablet d703e0b6c6cb433d9c49b5d7956ae360 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:56 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:57.398703 5255 raft_consensus.cc:515] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.400485 5255 leader_election.cc:290] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:57.402309 4198 tablet_service.cc:1511] Processing CreateTablet for tablet 96901efdb27047c0a48040bff11be8b4 (DEFAULT_TABLE table=default.Table_3ca34fb096c24aa692126c78e7d66dbd [id=57190c6f3f2d4d1a95337651dffd89ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:57.403401 4198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96901efdb27047c0a48040bff11be8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:57.407209 4201 tablet_service.cc:1511] Processing CreateTablet for tablet f9bf7670880746848933517216df3bf3 (DEFAULT_TABLE table=default.Table_cc310773560b43f89ee4742daa2360c6 [id=e40e56a9eaeb45d2be94d02eb6dbb5c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:57.408123 4201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9bf7670880746848933517216df3bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:57.290122 4776 tablet_bootstrap.cc:492] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:57.409566 4776 ts_tablet_manager.cc:1403] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.370s user 0.017s sys 0.009s
I20260430 19:58:57.411888 4776 raft_consensus.cc:359] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.412451 4776 raft_consensus.cc:385] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.412686 4776 raft_consensus.cc:740] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.413211 4776 consensus_queue.cc:260] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.420331 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf90db44540f2b438966361af8192" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:57.429693 4158 leader_election.cc:343] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:57.432678 5293 tablet.cc:1724] T 4764dc427b4a48309de914c4f5e655fe P 5a7356e99e744525861e12c108cfa3fe: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:57.444101 4729 tablet_bootstrap.cc:492] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:57.444634 4729 ts_tablet_manager.cc:1403] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.056s user 0.008s sys 0.002s
I20260430 19:58:57.447484 4776 ts_tablet_manager.cc:1434] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260430 19:58:57.448343 4776 tablet_bootstrap.cc:492] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:57.447525 4729 raft_consensus.cc:359] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.451539 4729 raft_consensus.cc:385] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.451882 4729 raft_consensus.cc:740] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.452682 4189 tablet_service.cc:1558] Processing DeleteTablet for tablet d846bbe4af0544e88ea5012ccc815881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:56 UTC) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:57.452620 4729 consensus_queue.cc:260] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.460414 4742 log.cc:1199] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/db0efaecfef44457a5128e878796c061
I20260430 19:58:57.461179 4742 ts_tablet_manager.cc:1950] T db0efaecfef44457a5128e878796c061 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:57.461561 5292 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:57.462546 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 176) took 3358 ms. Trace:
I20260430 19:58:57.462945 4742 rpcz_store.cc:276] 0430 19:58:54.103891 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.235809 (+131918us) service_pool.cc:225] Handling call
0430 19:58:56.829405 (+2593596us) ts_tablet_manager.cc:1151] Deleting tablet db0efaecfef44457a5128e878796c061
0430 19:58:56.829447 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:57.168183 (+338736us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:57.225700 (+ 57517us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:57.462495 (+236795us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4578,"tablet-delete.queue_time_us":2573928}
I20260430 19:58:57.454300 5294 tablet.cc:1724] T 4764dc427b4a48309de914c4f5e655fe P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:57.466784 5292 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:57.464828 4729 ts_tablet_manager.cc:1434] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260430 19:58:57.469650 3957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:57.469798 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:57.472513 4954 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:57.468866 5292 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:57.475893 5292 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:58:57.478003 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet db0efaecfef44457a5128e878796c061 (table default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c [id=d4bfa57fdfc64108ba1e909d0230fb51]) successfully deleted
I20260430 19:58:57.478291 4729 tablet_bootstrap.cc:492] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:57.487322 5266 raft_consensus.cc:493] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:57.487747 5266 raft_consensus.cc:515] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:57.489531 5266 leader_election.cc:290] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:57.493031 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3423379d1e3d4020a4241bf8917f1050" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:57.493805 4223 raft_consensus.cc:2393] T 3423379d1e3d4020a4241bf8917f1050 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:57.360421 4028 catalog_manager.cc:5671] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3). New cstate: current_term: 1 leader_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } }
I20260430 19:58:57.506523 4028 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:57.507277 4020 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_2b693cd1b0344bb4957052067f59f377 [id=2c86f02b8cd14685913a92ee25eb0483] alter complete (version 2)
W20260430 19:58:57.512432 4024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.1:59387 (request call id 58) took 897 ms (client timeout 1000 ms). Trace:
W20260430 19:58:57.512719 4024 rpcz_store.cc:269] 0430 19:58:56.615226 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:56.629863 (+ 14637us) service_pool.cc:225] Handling call
0430 19:58:57.512377 (+882514us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22870,"reported_tablets":8}
W20260430 19:58:57.520803 4020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.2:49905 (request call id 58) took 1047 ms (client timeout 1000 ms). Trace:
W20260430 19:58:57.521209 4020 rpcz_store.cc:269] 0430 19:58:56.473675 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:56.473775 (+ 100us) service_pool.cc:225] Handling call
0430 19:58:57.520737 (+1046962us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:58:56.620586 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:56.620957 (+ 371us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:56.620979 (+ 22us) write_op.cc:435] Acquired schema lock
0430 19:58:56.620991 (+ 12us) tablet.cc:662] Decoding operations
0430 19:58:56.621245 (+ 254us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:56.621278 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:56.621300 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:56.621501 (+ 201us) tablet.cc:701] Row locks acquired
0430 19:58:56.621515 (+ 14us) write_op.cc:260] PREPARE: finished
0430 19:58:56.621620 (+ 105us) write_op.cc:270] Start()
0430 19:58:56.621699 (+ 79us) write_op.cc:276] Timestamp: P: 1777579136621382 usec, L: 0
0430 19:58:56.621715 (+ 16us) op_driver.cc:348] REPLICATION: starting
0430 19:58:56.622211 (+ 496us) log.cc:844] Serialized 527 byte log entry
0430 19:58:56.933280 (+311069us) op_driver.cc:464] REPLICATION: finished
0430 19:58:56.934520 (+ 1240us) write_op.cc:301] APPLY: starting
0430 19:58:56.934580 (+ 60us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:56.935311 (+ 731us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:56.935392 (+ 81us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:56.936181 (+ 789us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:56.936305 (+ 124us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:56.936344 (+ 39us) write_op.cc:312] APPLY: finished
0430 19:58:56.936512 (+ 168us) log.cc:844] Serialized 23 byte log entry
0430 19:58:56.936654 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:56.936711 (+ 57us) write_op.cc:454] Released schema lock
0430 19:58:56.936739 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:56.937123 (+ 384us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:56.937161 (+ 38us) write_op.cc:454] Released schema lock
Related trace 'op':
0430 19:58:57.184562 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:58:57.184925 (+ 363us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:58:57.184948 (+ 23us) write_op.cc:435] Acquired schema lock
0430 19:58:57.184961 (+ 13us) tablet.cc:662] Decoding operations
0430 19:58:57.185199 (+ 238us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:58:57.185234 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0430 19:58:57.185255 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0430 19:58:57.185379 (+ 124us) tablet.cc:701] Row locks acquired
0430 19:58:57.185391 (+ 12us) write_op.cc:260] PREPARE: finished
0430 19:58:57.185492 (+ 101us) write_op.cc:270] Start()
0430 19:58:57.185570 (+ 78us) write_op.cc:276] Timestamp: P: 1777579137185397 usec, L: 0
0430 19:58:57.185586 (+ 16us) op_driver.cc:348] REPLICATION: starting
0430 19:58:57.186032 (+ 446us) log.cc:844] Serialized 356 byte log entry
0430 19:58:57.367103 (+181071us) op_driver.cc:464] REPLICATION: finished
0430 19:58:57.369024 (+ 1921us) write_op.cc:301] APPLY: starting
0430 19:58:57.369080 (+ 56us) tablet.cc:1367] starting BulkCheckPresence
0430 19:58:57.369158 (+ 78us) tablet.cc:1370] finished BulkCheckPresence
0430 19:58:57.369238 (+ 80us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:58:57.369405 (+ 167us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:58:57.369521 (+ 116us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0430 19:58:57.369563 (+ 42us) write_op.cc:312] APPLY: finished
0430 19:58:57.369726 (+ 163us) log.cc:844] Serialized 23 byte log entry
0430 19:58:57.369860 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:57.369910 (+ 50us) write_op.cc:454] Released schema lock
0430 19:58:57.369940 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0430 19:58:57.370133 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:58:57.370159 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":1098,"apply.run_cpu_time_us":1328,"apply.run_wall_time_us":2522,"num_ops":1,"prepare.queue_time_us":97,"prepare.run_cpu_time_us":1825,"prepare.run_wall_time_us":1825,"replication_time_us":311479}],["op",{"apply.queue_time_us":1809,"apply.run_cpu_time_us":1084,"apply.run_wall_time_us":1084,"num_ops":1,"prepare.queue_time_us":121576,"prepare.run_cpu_time_us":1672,"prepare.run_wall_time_us":1672,"replication_time_us":181444}]]}
I20260430 19:58:57.525629 4729 tablet_bootstrap.cc:654] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:57.532795 4203 tablet_service.cc:1558] Processing DeleteTablet for tablet 3423379d1e3d4020a4241bf8917f1050 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f0489b4631346f4952ef648e0653242) from {username='slave'} at 127.0.0.1:41996
I20260430 19:58:57.531634 4027 master_service.cc:954] ReplaceTablet: received request to replace tablet 619fb8dd24244d1ea81953668659df8a from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:57.555874 4729 tablet_bootstrap.cc:492] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:57.556278 4729 ts_tablet_manager.cc:1403] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.078s user 0.011s sys 0.005s
I20260430 19:58:57.558478 4729 raft_consensus.cc:359] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.559089 4729 raft_consensus.cc:385] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.559312 4729 raft_consensus.cc:740] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.559967 4729 consensus_queue.cc:260] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.570043 4729 ts_tablet_manager.cc:1434] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260430 19:58:57.578902 5266 raft_consensus.cc:493] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:57.579341 5266 raft_consensus.cc:515] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.580786 4026 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_c9c47d2286e64a108c57b78454f22006" } modify_external_catalogs: true
I20260430 19:58:57.581157 5266 leader_election.cc:290] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:57.581301 4026 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55130:
table { table_name: "default.Table_c9c47d2286e64a108c57b78454f22006" } modify_external_catalogs: true
I20260430 19:58:57.583276 4020 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_82450907cdb2440796c7ca77026e738a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:57.584707 4020 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82450907cdb2440796c7ca77026e738a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:57.585511 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:57.586053 4223 raft_consensus.cc:2468] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:57.586989 4470 leader_election.cc:304] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:57.587687 5266 raft_consensus.cc:2804] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:57.587996 5266 raft_consensus.cc:493] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:57.588303 5266 raft_consensus.cc:3060] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.591033 3832 master-stress-test.cc:577] Killing test thread 1/10
I20260430 19:58:57.595249 4729 tablet_bootstrap.cc:492] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:57.600940 5266 raft_consensus.cc:515] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.615088 4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:57.615751 4223 raft_consensus.cc:3060] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.464612 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 174) took 3361 ms. Trace:
I20260430 19:58:57.622161 5266 leader_election.cc:290] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:57.622219 4742 rpcz_store.cc:276] 0430 19:58:54.103287 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.235834 (+132547us) service_pool.cc:225] Handling call
0430 19:58:57.464299 (+3228465us) ts_tablet_manager.cc:1151] Deleting tablet 4758b3d846434b2b8466033c083b7339
0430 19:58:57.464337 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:57.464565 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3173281}
I20260430 19:58:57.604768 4729 tablet_bootstrap.cc:654] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:57.460042 4776 tablet_bootstrap.cc:654] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:57.633437 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 4758b3d846434b2b8466033c083b7339 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4758b3d846434b2b8466033c083b7339 already in progress: creating tablet
W20260430 19:58:57.198768 4320 leader_election.cc:343] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:57.637617 4320 leader_election.cc:304] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:57.621764 4223 raft_consensus.cc:2468] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:57.641055 4470 leader_election.cc:304] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f; no voters:
I20260430 19:58:57.651072 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:57.651810 4531 raft_consensus.cc:2393] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8345ec7f57c4399a8a4afe4dd43439e in current term 1: Already voted for candidate 8a11a9a7f244435b8e84554e3fabbd0f in this term.
I20260430 19:58:57.653808 5191 raft_consensus.cc:2804] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:57.654245 5191 raft_consensus.cc:697] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:57.655020 5191 consensus_queue.cc:237] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.657161 5198 consensus_queue.cc:1048] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:57.669860 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 179) took 3565 ms. Trace:
I20260430 19:58:57.670192 4742 rpcz_store.cc:276] 0430 19:58:54.104853 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.236157 (+131304us) service_pool.cc:225] Handling call
0430 19:58:57.634364 (+3398207us) ts_tablet_manager.cc:1151] Deleting tablet 34a6d662493e428aa475979abbffc9d7
0430 19:58:57.669541 (+ 35177us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:57.669808 (+ 267us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3339349}
I20260430 19:58:57.671525 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 175) took 3568 ms. Trace:
I20260430 19:58:57.671818 4742 rpcz_store.cc:276] 0430 19:58:54.103578 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.235253 (+131675us) service_pool.cc:225] Handling call
0430 19:58:57.671221 (+3435968us) ts_tablet_manager.cc:1151] Deleting tablet f5c54c37bdae458d88594a3f5c401a48
0430 19:58:57.671259 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:57.671476 (+ 217us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3361632}
I20260430 19:58:57.672919 4742 tablet_replica.cc:333] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:57.680192 5289 raft_consensus.cc:493] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:57.680972 5289 raft_consensus.cc:515] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:57.682694 5092 raft_consensus.cc:493] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:57.683159 5092 raft_consensus.cc:515] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.684921 5092 leader_election.cc:290] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:57.685405 5092 raft_consensus.cc:2749] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 19:58:57.685875 4029 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 95b78965508247cface36a104d1ead9f has no consensus state
I20260430 19:58:57.662794 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf90db44540f2b438966361af8192" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:57.688211 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "443308f2a26f4622aa4551ff5f73fc39" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:57.688769 4531 raft_consensus.cc:2468] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
W20260430 19:58:57.691622 4022 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e9927db780674f31be8f043fe0843736 consensus state has no leader
I20260430 19:58:57.697083 4317 leader_election.cc:304] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:57.698079 5174 raft_consensus.cc:2804] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:57.686353 4384 raft_consensus.cc:2468] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 0.
W20260430 19:58:57.674371 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet 34a6d662493e428aa475979abbffc9d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34a6d662493e428aa475979abbffc9d7 already in progress: creating tablet
I20260430 19:58:57.698436 5174 raft_consensus.cc:493] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260430 19:58:57.700452 4016 catalog_manager.cc:4982] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): delete failed for tablet f5c54c37bdae458d88594a3f5c401a48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5c54c37bdae458d88594a3f5c401a48 already in progress: creating tablet
I20260430 19:58:57.702726 4161 leader_election.cc:304] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:57.703827 5288 raft_consensus.cc:2804] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:57.704170 5288 raft_consensus.cc:493] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:57.704480 5288 raft_consensus.cc:3060] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.673638 4742 raft_consensus.cc:2243] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:57.717865 5198 consensus_queue.cc:1048] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:57.720219 4380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3423379d1e3d4020a4241bf8917f1050" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:57.700645 5174 raft_consensus.cc:3060] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.732826 4742 raft_consensus.cc:2272] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:57.738202 4818 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.662s user 0.036s sys 0.057s
W20260430 19:58:57.738636 4818 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.663s user 0.036s sys 0.058s
I20260430 19:58:57.740671 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce21ac617c5b4d6ca97c6f68f53c6a67" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:57.739445 5289 leader_election.cc:290] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
W20260430 19:58:57.742846 4317 leader_election.cc:343] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:57.745296 4015 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet e9927db780674f31be8f043fe0843736 (table default.Table_84d9207cf43849148c90cd0b2fa6c17f [id=54576f526a0c449e90b1e8c86326b803], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e9927db780674f31be8f043fe0843736 consensus state has no leader
I20260430 19:58:57.717852 5288 raft_consensus.cc:515] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.769937 5288 leader_election.cc:290] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:57.755442 4776 tablet_bootstrap.cc:492] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:57.771682 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf90db44540f2b438966361af8192" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:57.771956 4776 ts_tablet_manager.cc:1403] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.324s user 0.005s sys 0.005s
W20260430 19:58:57.773903 4158 leader_election.cc:343] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:57.774202 4776 raft_consensus.cc:359] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.776860 4021 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55130:
name: "default.Table_30b983616c0f420a9f8173d25d11026c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:58:57.778548 4021 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30b983616c0f420a9f8173d25d11026c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:58:57.779484 4818 tablet_bootstrap.cc:492] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:57.780009 4818 ts_tablet_manager.cc:1403] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.880s user 0.043s sys 0.062s
I20260430 19:58:57.780164 4380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf90db44540f2b438966361af8192" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:57.783636 4818 raft_consensus.cc:359] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:57.784440 4818 raft_consensus.cc:385] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.785022 4818 raft_consensus.cc:740] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.784732 4380 raft_consensus.cc:3060] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.791700 4380 raft_consensus.cc:2468] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a7356e99e744525861e12c108cfa3fe in term 1.
I20260430 19:58:57.810737 4023 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 865707fe4ad24dfe9d8305a3e4e00529
I20260430 19:58:57.794276 4383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:57.800163 4384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:57.814905 4383 raft_consensus.cc:2393] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:57.817245 4384 raft_consensus.cc:2393] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
I20260430 19:58:57.824571 4470 leader_election.cc:304] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:57.826560 5198 raft_consensus.cc:3060] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:57.831015 4356 consensus_meta.cc:287] T 608d010d0acc44b3bc06e0ccf9674bc0 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent flushing consensus metadata: real 0.588s user 0.006s sys 0.004s
I20260430 19:58:57.834844 4026 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 43e38b2e10d54be288b0035b69ffee14
I20260430 19:58:57.835029 5117 ts_tablet_manager.cc:1929] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:57.835419 5117 log.cc:1199] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/09f272340cd942e9b2af9e38a2bdec82
I20260430 19:58:57.836162 5117 ts_tablet_manager.cc:1950] T 09f272340cd942e9b2af9e38a2bdec82 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:57.801033 4818 consensus_queue.cc:260] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:57.839072 4818 ts_tablet_manager.cc:1434] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.059s user 0.001s sys 0.006s
I20260430 19:58:57.839597 5198 raft_consensus.cc:2749] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:57.844105 4818 tablet_bootstrap.cc:492] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:57.804545 5174 raft_consensus.cc:515] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.849028 4531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "443308f2a26f4622aa4551ff5f73fc39" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:57.849536 4531 raft_consensus.cc:3060] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.856349 4531 raft_consensus.cc:2468] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:57.867302 3832 master-stress-test.cc:579] Killed test thread 1/10
I20260430 19:58:57.867579 3832 master-stress-test.cc:577] Killing test thread 2/10
I20260430 19:58:57.867915 3832 master-stress-test.cc:579] Killed test thread 2/10
I20260430 19:58:57.868151 3832 master-stress-test.cc:577] Killing test thread 3/10
I20260430 19:58:57.868368 3832 master-stress-test.cc:579] Killed test thread 3/10
I20260430 19:58:57.868541 3832 master-stress-test.cc:577] Killing test thread 4/10
I20260430 19:58:57.868726 3832 master-stress-test.cc:579] Killed test thread 4/10
I20260430 19:58:57.868983 3832 master-stress-test.cc:577] Killing test thread 5/10
I20260430 19:58:57.869197 3832 master-stress-test.cc:579] Killed test thread 5/10
I20260430 19:58:57.869370 3832 master-stress-test.cc:577] Killing test thread 6/10
I20260430 19:58:57.869802 3832 master-stress-test.cc:579] Killed test thread 6/10
I20260430 19:58:57.870013 3832 master-stress-test.cc:577] Killing test thread 7/10
I20260430 19:58:57.870240 3832 master-stress-test.cc:579] Killed test thread 7/10
I20260430 19:58:57.870407 3832 master-stress-test.cc:577] Killing test thread 8/10
I20260430 19:58:57.870718 3832 master-stress-test.cc:579] Killed test thread 8/10
I20260430 19:58:57.870914 3832 master-stress-test.cc:577] Killing test thread 9/10
I20260430 19:58:57.877166 4729 tablet_bootstrap.cc:492] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:57.877569 4729 ts_tablet_manager.cc:1403] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.283s user 0.007s sys 0.006s
I20260430 19:58:57.879752 4317 leader_election.cc:304] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
W20260430 19:58:57.879858 4320 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 681025us
I20260430 19:58:57.880666 5289 raft_consensus.cc:2804] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:57.886735 4776 raft_consensus.cc:385] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.886786 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce21ac617c5b4d6ca97c6f68f53c6a67" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:57.887174 4776 raft_consensus.cc:740] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.887799 4776 consensus_queue.cc:260] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.852234 4742 ts_tablet_manager.cc:1916] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:57.889819 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 3423379d1e3d4020a4241bf8917f1050 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f0489b4631346f4952ef648e0653242) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:57.891523 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet db0efaecfef44457a5128e878796c061 with delete_type TABLET_DATA_DELETED (replaced by tablet d0c2c08fac7248a1b3b52e2b43041dff) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:57.892729 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 081a060c44924bcda09ffc8fe88012b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:57.893079 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 70d410f3b3b04e649b79dda04c0891ee with delete_type TABLET_DATA_DELETED (replaced by tablet b0d0ac18ef9040c8a5bd804178081083) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:57.893414 5247 consensus_queue.cc:1048] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:57.894234 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet b6acec54bc7b4692894333e0150a1aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:57.884034 5288 raft_consensus.cc:493] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:57.886142 4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "443308f2a26f4622aa4551ff5f73fc39" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:57.894779 5092 consensus_queue.cc:1048] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:57.894985 5288 raft_consensus.cc:515] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
W20260430 19:58:57.895861 4320 leader_election.cc:343] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:57.895834 4224 raft_consensus.cc:2468] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:57.896193 4320 leader_election.cc:304] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:57.897275 5247 consensus_queue.cc:1048] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:57.897900 5092 raft_consensus.cc:2749] T ce21ac617c5b4d6ca97c6f68f53c6a67 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:57.883383 5289 raft_consensus.cc:697] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:57.899714 5289 consensus_queue.cc:237] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } }
I20260430 19:58:57.900624 5288 leader_election.cc:290] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:57.890574 4356 tablet_service.cc:1511] Processing CreateTablet for tablet f9bf7670880746848933517216df3bf3 (DEFAULT_TABLE table=default.Table_cc310773560b43f89ee4742daa2360c6 [id=e40e56a9eaeb45d2be94d02eb6dbb5c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:57.901701 4530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "443308f2a26f4622aa4551ff5f73fc39" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:57.902624 4530 raft_consensus.cc:2393] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a7356e99e744525861e12c108cfa3fe in current term 1: Already voted for candidate d8345ec7f57c4399a8a4afe4dd43439e in this term.
W20260430 19:58:57.903898 4320 consensus_peers.cc:597] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:57.905905 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6acec54bc7b4692894333e0150a1aff" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:57.906663 4222 raft_consensus.cc:2393] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8345ec7f57c4399a8a4afe4dd43439e in current term 1: Already voted for candidate 8a11a9a7f244435b8e84554e3fabbd0f in this term.
I20260430 19:58:57.907579 4320 leader_election.cc:304] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:57.913374 5174 leader_election.cc:290] T 443308f2a26f4622aa4551ff5f73fc39 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:57.916728 4379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "443308f2a26f4622aa4551ff5f73fc39" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:57.919909 5174 raft_consensus.cc:2749] T b6acec54bc7b4692894333e0150a1aff P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260430 19:58:57.921551 4021 master_service.cc:954] ReplaceTablet: received request to replace tablet 6f39fb64a301447f988c53ccfe896074 from {username='slave'} at 127.0.0.1:55130
I20260430 19:58:57.921645 4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "443308f2a26f4622aa4551ff5f73fc39" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe"
I20260430 19:58:57.922518 4161 leader_election.cc:304] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:57.926817 4222 raft_consensus.cc:3060] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:57.930835 5266 consensus_queue.cc:1048] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:57.934399 4510 tablet_service.cc:1558] Processing DeleteTablet for tablet 70d410f3b3b04e649b79dda04c0891ee with delete_type TABLET_DATA_DELETED (replaced by tablet b0d0ac18ef9040c8a5bd804178081083) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:57.935441 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet 395f97fdf3834cb9aaabd261e1bf76c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:57.936975 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet db0efaecfef44457a5128e878796c061 with delete_type TABLET_DATA_DELETED (replaced by tablet d0c2c08fac7248a1b3b52e2b43041dff) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:57.939932 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet 081a060c44924bcda09ffc8fe88012b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:51 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:57.943075 3832 master-stress-test.cc:579] Killed test thread 9/10
I20260430 19:58:57.943261 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9bf7670880746848933517216df3bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:57.943341 3832 master-stress-test.cc:577] Killing test thread 10/10
I20260430 19:58:57.949026 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 181) took 3605 ms. Trace:
I20260430 19:58:57.949492 5117 rpcz_store.cc:276] 0430 19:58:54.343637 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.343838 (+ 201us) service_pool.cc:225] Handling call
0430 19:58:57.122698 (+2778860us) ts_tablet_manager.cc:1151] Deleting tablet 09f272340cd942e9b2af9e38a2bdec82
0430 19:58:57.122740 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:57.176918 (+ 54178us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:57.834807 (+657889us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:57.948954 (+114147us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2778349}
I20260430 19:58:57.951049 5117 tablet_replica.cc:333] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:57.951773 5117 raft_consensus.cc:2243] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:57.952771 5117 raft_consensus.cc:2272] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:57.953433 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 09f272340cd942e9b2af9e38a2bdec82 (table default.Table_276bd93958054b6b9ad30f314bbb487c [id=de548c1ed3d8424b9a2d3c451444462c]) successfully deleted
I20260430 19:58:57.962949 4729 raft_consensus.cc:359] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.963577 4729 raft_consensus.cc:385] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:57.963832 4729 raft_consensus.cc:740] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:57.970376 4222 raft_consensus.cc:2468] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
I20260430 19:58:57.973734 5313 tablet_replica.cc:333] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:57.974578 5313 raft_consensus.cc:2243] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:57.975412 5313 pending_rounds.cc:70] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Trying to abort 1 pending ops.
I20260430 19:58:57.975636 5313 pending_rounds.cc:77] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964140296929280 op_type: NO_OP noop_request { }
I20260430 19:58:57.975965 5313 raft_consensus.cc:2889] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:57.976279 5313 raft_consensus.cc:2272] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:57.971930 4776 ts_tablet_manager.cc:1434] T f5c54c37bdae458d88594a3f5c401a48 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.200s user 0.004s sys 0.001s
I20260430 19:58:57.978252 5313 ts_tablet_manager.cc:1916] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:58:57.974891 4161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.2:43939, user_credentials={real_user=slave}} blocked reactor thread for 52449.9us
I20260430 19:58:57.979009 4161 leader_election.cc:304] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e; no voters: 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:57.979972 4776 tablet_bootstrap.cc:492] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:57.976399 5284 raft_consensus.cc:2749] T 443308f2a26f4622aa4551ff5f73fc39 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:57.985082 5284 raft_consensus.cc:2804] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:57.985522 5284 raft_consensus.cc:697] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [term 1 LEADER]: Becoming Leader. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Running, Role: LEADER
I20260430 19:58:57.995172 4729 consensus_queue.cc:260] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:57.986234 5284 consensus_queue.cc:237] T ba4cf90db44540f2b438966361af8192 P 5a7356e99e744525861e12c108cfa3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.007714 4729 ts_tablet_manager.cc:1434] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.130s user 0.006s sys 0.000s
I20260430 19:58:58.023293 5117 ts_tablet_manager.cc:1916] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.030225 4818 tablet_bootstrap.cc:654] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.035333 4776 tablet_bootstrap.cc:654] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.044374 4742 ts_tablet_manager.cc:1929] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.050662 5288 raft_consensus.cc:493] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:58.054831 5288 raft_consensus.cc:515] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.056058 5174 raft_consensus.cc:493] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:58.056644 5288 leader_election.cc:290] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:58.056550 5174 raft_consensus.cc:515] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.059706 4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1f7489f1d9c46b8b33745c3e03a2138" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7356e99e744525861e12c108cfa3fe" is_pre_election: true
I20260430 19:58:58.060891 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ba8046c294be3ac20e2d93925d17b" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:58.061335 4320 leader_election.cc:343] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Illegal state: must be running to vote when last-logged opid is not known
W20260430 19:58:58.062083 4158 leader_election.cc:343] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:58.062939 4742 log.cc:1199] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/eafc058f392a45de9d30b521ebde0950
I20260430 19:58:58.063771 4742 ts_tablet_manager.cc:1950] T eafc058f392a45de9d30b521ebde0950 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:58.066169 4729 tablet_bootstrap.cc:492] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:58.068353 4776 tablet_bootstrap.cc:492] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe: No bootstrap required, opened a new log
I20260430 19:58:58.070371 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 180) took 3650 ms. Trace:
I20260430 19:58:58.070873 4742 rpcz_store.cc:276] 0430 19:58:54.419692 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.420966 (+ 1274us) service_pool.cc:225] Handling call
0430 19:58:57.672776 (+3251810us) ts_tablet_manager.cc:1151] Deleting tablet eafc058f392a45de9d30b521ebde0950
0430 19:58:57.672812 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.017182 (+344370us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.044259 (+ 27077us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.070316 (+ 26057us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3251211}
I20260430 19:58:58.074414 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet eafc058f392a45de9d30b521ebde0950 (table default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e [id=8d1c3bb61831442596e2112cef9032a8]) successfully deleted
I20260430 19:58:58.075482 4742 tablet_replica.cc:333] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:58.076215 4742 raft_consensus.cc:2243] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.076730 4742 raft_consensus.cc:2272] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.071472 4776 ts_tablet_manager.cc:1403] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe: Time spent bootstrapping tablet: real 0.092s user 0.010s sys 0.001s
I20260430 19:58:58.077778 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1f7489f1d9c46b8b33745c3e03a2138" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:58.079893 4317 leader_election.cc:343] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:58.079605 4776 raft_consensus.cc:359] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.080343 4776 raft_consensus.cc:385] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.080307 4317 leader_election.cc:304] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8345ec7f57c4399a8a4afe4dd43439e; no voters: 5a7356e99e744525861e12c108cfa3fe, 8a11a9a7f244435b8e84554e3fabbd0f
I20260430 19:58:58.080570 4776 raft_consensus.cc:740] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7356e99e744525861e12c108cfa3fe, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.081068 5092 raft_consensus.cc:2749] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:58.077030 4382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ba8046c294be3ac20e2d93925d17b" candidate_uuid: "5a7356e99e744525861e12c108cfa3fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:58.075286 5174 leader_election.cc:290] T d1f7489f1d9c46b8b33745c3e03a2138 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
W20260430 19:58:58.082674 4161 leader_election.cc:343] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:58.083022 4161 leader_election.cc:304] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7356e99e744525861e12c108cfa3fe; no voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:58.083606 5288 raft_consensus.cc:2749] T 958ba8046c294be3ac20e2d93925d17b P 5a7356e99e744525861e12c108cfa3fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:58.085837 4742 ts_tablet_manager.cc:1916] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.089330 4729 tablet_bootstrap.cc:654] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.084674 5318 tablet.cc:1724] T 4764dc427b4a48309de914c4f5e655fe P d8345ec7f57c4399a8a4afe4dd43439e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260430 19:58:58.096586 4776 consensus_queue.cc:260] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.098855 4776 ts_tablet_manager.cc:1434] T 28ba03500c0540cda44debe362278e47 P 5a7356e99e744525861e12c108cfa3fe: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260430 19:58:58.107316 4776 tablet_bootstrap.cc:492] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 5a7356e99e744525861e12c108cfa3fe: Bootstrap starting.
I20260430 19:58:58.115701 4818 tablet_bootstrap.cc:492] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:58.116334 4818 ts_tablet_manager.cc:1403] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.272s user 0.011s sys 0.004s
I20260430 19:58:58.121183 4776 tablet_bootstrap.cc:654] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 5a7356e99e744525861e12c108cfa3fe: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.148802 4742 ts_tablet_manager.cc:1929] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.149197 4742 log.cc:1199] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/081a060c44924bcda09ffc8fe88012b3
I20260430 19:58:58.155205 4742 ts_tablet_manager.cc:1950] T 081a060c44924bcda09ffc8fe88012b3 P 5a7356e99e744525861e12c108cfa3fe: Deleting consensus metadata
I20260430 19:58:58.154821 4818 raft_consensus.cc:359] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.157595 4818 raft_consensus.cc:385] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.157912 4818 raft_consensus.cc:740] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.158651 4818 consensus_queue.cc:260] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.164803 5117 ts_tablet_manager.cc:1929] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.165269 5117 log.cc:1199] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/eafc058f392a45de9d30b521ebde0950
I20260430 19:58:58.166098 5117 ts_tablet_manager.cc:1950] T eafc058f392a45de9d30b521ebde0950 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:58.167255 4742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41996 (request call id 182) took 3636 ms. Trace:
I20260430 19:58:58.176846 4027 catalog_manager.cc:5958] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: Sending DeleteTablet for 3 replicas of tablet 619fb8dd24244d1ea81953668659df8a
I20260430 19:58:58.176923 4742 rpcz_store.cc:276] 0430 19:58:54.531069 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.578105 (+ 47036us) service_pool.cc:225] Handling call
0430 19:58:58.075341 (+3497236us) ts_tablet_manager.cc:1151] Deleting tablet 081a060c44924bcda09ffc8fe88012b3
0430 19:58:58.075383 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.103289 (+ 27906us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.148677 (+ 45388us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.167198 (+ 18521us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3430612}
I20260430 19:58:58.177942 4742 tablet_replica.cc:333] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe: stopping tablet replica
I20260430 19:58:58.178647 4742 raft_consensus.cc:2243] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.178440 4510 tablet_service.cc:1558] Processing DeleteTablet for tablet 619fb8dd24244d1ea81953668659df8a with delete_type TABLET_DATA_DELETED (replaced by tablet a6f8fc401a504932b3a204adbdecc9bc) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:58.179077 4742 raft_consensus.cc:2272] T b6acec54bc7b4692894333e0150a1aff P 5a7356e99e744525861e12c108cfa3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.180058 4027 catalog_manager.cc:6741] ReplaceTablet: tablet 619fb8dd24244d1ea81953668659df8a deleted and replaced by tablet a6f8fc401a504932b3a204adbdecc9bc
I20260430 19:58:58.182853 4818 ts_tablet_manager.cc:1434] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.066s user 0.005s sys 0.000s
I20260430 19:58:58.183876 4818 tablet_bootstrap.cc:492] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:58.184288 3832 master-stress-test.cc:579] Killed test thread 10/10
I20260430 19:58:58.184662 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 4126
I20260430 19:58:58.185968 4016 catalog_manager.cc:5002] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): tablet 081a060c44924bcda09ffc8fe88012b3 (table default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]) successfully deleted
I20260430 19:58:58.187168 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 619fb8dd24244d1ea81953668659df8a with delete_type TABLET_DATA_DELETED (replaced by tablet a6f8fc401a504932b3a204adbdecc9bc) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.204180 4024 catalog_manager.cc:5763] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065) does not have the latest schema for tablet 958ba8046c294be3ac20e2d93925d17b (table default.Table_7b77d87732e142a7b0687c71ce62d0c3 [id=ffb188d244694ce2aa7985340abb8813]). Expected version 1 got 0
I20260430 19:58:58.209595 4021 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 4764dc427b4a48309de914c4f5e655fe (table default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3]). Expected version 2 got 0
I20260430 19:58:58.209743 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 182) took 3849 ms. Trace:
I20260430 19:58:58.210052 5117 rpcz_store.cc:276] 0430 19:58:54.360490 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.360610 (+ 120us) service_pool.cc:225] Handling call
0430 19:58:57.950889 (+3590279us) ts_tablet_manager.cc:1151] Deleting tablet eafc058f392a45de9d30b521ebde0950
0430 19:58:57.950951 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.064616 (+113665us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.164677 (+100061us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.209663 (+ 44986us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40015,"tablet-delete.queue_time_us":3589759}
I20260430 19:58:58.211625 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet eafc058f392a45de9d30b521ebde0950 (table default.Table_89cd3b16c31b42fcb83dc3bcbb951d9e [id=8d1c3bb61831442596e2112cef9032a8]) successfully deleted
I20260430 19:58:58.212621 5117 tablet_replica.cc:333] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:58.213315 5117 raft_consensus.cc:2243] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.213788 5117 raft_consensus.cc:2272] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.215759 5117 ts_tablet_manager.cc:1916] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.225812 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet ba4cf90db44540f2b438966361af8192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.237257 5117 ts_tablet_manager.cc:1929] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:58.237674 5117 log.cc:1199] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/e4e87fda401047cfa7070ae95b1701a9
I20260430 19:58:58.238291 4818 tablet_bootstrap.cc:654] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.238951 5117 ts_tablet_manager.cc:1950] T e4e87fda401047cfa7070ae95b1701a9 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:58.243031 4510 tablet_service.cc:1511] Processing CreateTablet for tablet 63242908d9a94becbb83e99be1dfbdfd (DEFAULT_TABLE table=default.Table_d5e01da3c4624e18a5371fb5388caea4 [id=927ff067803742feb754854af9078f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.244251 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63242908d9a94becbb83e99be1dfbdfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.244930 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 186) took 3717 ms. Trace:
I20260430 19:58:58.247629 5117 rpcz_store.cc:276] 0430 19:58:54.527807 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.527943 (+ 136us) service_pool.cc:225] Handling call
0430 19:58:58.212461 (+3684518us) ts_tablet_manager.cc:1151] Deleting tablet e4e87fda401047cfa7070ae95b1701a9
0430 19:58:58.212515 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.230702 (+ 18187us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.237141 (+ 6439us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.244868 (+ 7727us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3684002}
I20260430 19:58:58.250785 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 193) took 3492 ms. Trace:
I20260430 19:58:58.252058 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet e4e87fda401047cfa7070ae95b1701a9 (table default.Table_4b6fc137cf6945079fa8310db94a2b1a [id=2cb52e96af9f4eea998718eef0ac58ad]) successfully deleted
I20260430 19:58:58.254199 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ba03500c0540cda44debe362278e47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.254186 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet eafc058f392a45de9d30b521ebde0950 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:54 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.254693 5117 rpcz_store.cc:276] 0430 19:58:54.758616 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.758743 (+ 127us) service_pool.cc:225] Handling call
0430 19:58:58.250401 (+3491658us) ts_tablet_manager.cc:1151] Deleting tablet ea907f9bd93041de976a2f0c3b517dab
0430 19:58:58.250458 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.250735 (+ 277us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3475342}
I20260430 19:58:58.254351 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 with delete_type TABLET_DATA_DELETED (replaced by tablet 70e4d5af75d04f6d9091d2b086e4a6ba) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.259434 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 207) took 3272 ms. Trace:
I20260430 19:58:58.259902 4509 tablet_service.cc:1511] Processing CreateTablet for tablet 5d93cb1b53dd4681a7b1b90a2d6b6bdd (DEFAULT_TABLE table=default.Table_79f86b4f9880405db20eade1a857c8e8 [id=2385c0800a004ed8bd6ffc2e8db368f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.260953 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d93cb1b53dd4681a7b1b90a2d6b6bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.259770 5117 rpcz_store.cc:276] 0430 19:58:54.986971 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:54.987103 (+ 132us) service_pool.cc:225] Handling call
0430 19:58:58.255561 (+3268458us) ts_tablet_manager.cc:1151] Deleting tablet 6f0489b4631346f4952ef648e0653242
0430 19:58:58.259067 (+ 3506us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.259379 (+ 312us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3258425}
I20260430 19:58:58.267115 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 211) took 2811 ms. Trace:
I20260430 19:58:58.267427 5117 rpcz_store.cc:276] 0430 19:58:55.455881 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:55.498412 (+ 42531us) service_pool.cc:225] Handling call
0430 19:58:58.266742 (+2768330us) ts_tablet_manager.cc:1151] Deleting tablet 84b02b29777341d3934f6a77967d79b0
0430 19:58:58.266796 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.267061 (+ 265us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2757873}
I20260430 19:58:58.269208 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 63242908d9a94becbb83e99be1dfbdfd (DEFAULT_TABLE table=default.Table_d5e01da3c4624e18a5371fb5388caea4 [id=927ff067803742feb754854af9078f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.270326 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63242908d9a94becbb83e99be1dfbdfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.271941 4818 tablet_bootstrap.cc:492] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:58.272336 4818 ts_tablet_manager.cc:1403] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.089s user 0.006s sys 0.006s
W20260430 19:58:58.274015 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet ea907f9bd93041de976a2f0c3b517dab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea907f9bd93041de976a2f0c3b517dab already in progress: creating tablet
W20260430 19:58:58.274691 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 6f0489b4631346f4952ef648e0653242 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f0489b4631346f4952ef648e0653242 already in progress: creating tablet
W20260430 19:58:58.275357 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 84b02b29777341d3934f6a77967d79b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84b02b29777341d3934f6a77967d79b0 already in progress: creating tablet
I20260430 19:58:58.283051 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 212) took 2690 ms. Trace:
I20260430 19:58:58.283429 4021 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 958ba8046c294be3ac20e2d93925d17b (table default.Table_7b77d87732e142a7b0687c71ce62d0c3 [id=ffb188d244694ce2aa7985340abb8813]). Expected version 1 got 0
I20260430 19:58:58.283397 5117 rpcz_store.cc:276] 0430 19:58:55.593136 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:55.593611 (+ 475us) service_pool.cc:225] Handling call
0430 19:58:58.272802 (+2679191us) ts_tablet_manager.cc:1151] Deleting tablet 3d36c2538ec84e0889c4e53975a2ff1c
0430 19:58:58.282685 (+ 9883us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.282992 (+ 307us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2674025}
I20260430 19:58:58.284345 5117 tablet_replica.cc:333] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:58.285072 5117 raft_consensus.cc:2243] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:58.285995 5117 raft_consensus.cc:2272] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.268692 5313 ts_tablet_manager.cc:1929] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.287029 5313 log.cc:1199] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/70d410f3b3b04e649b79dda04c0891ee
I20260430 19:58:58.287810 5313 ts_tablet_manager.cc:1950] T 70d410f3b3b04e649b79dda04c0891ee P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:58.289516 4818 raft_consensus.cc:359] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.290118 4818 raft_consensus.cc:385] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.290375 4818 raft_consensus.cc:740] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.291263 4028 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_06eacb8ab2f84a0fb4d751f4190d8f0f [id=4b4c0010ad354dd28be8cc7142d321c3] alter complete (version 2)
I20260430 19:58:58.291453 4818 consensus_queue.cc:260] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
W20260430 19:58:58.294411 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 3d36c2538ec84e0889c4e53975a2ff1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d36c2538ec84e0889c4e53975a2ff1c already in progress: creating tablet
I20260430 19:58:58.296784 4818 ts_tablet_manager.cc:1434] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
W20260430 19:58:58.294123 4024 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.190.1:59387 (request call id 59) took 765 ms (client timeout 1000 ms). Trace:
I20260430 19:58:58.296913 5313 tablet_replica.cc:333] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:58.296921 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 70d410f3b3b04e649b79dda04c0891ee (table default.Table_38209ef2ff664e2482f5795be37d93f4 [id=67fa374bac9d4528b3f9a93e75558678]) successfully deleted
I20260430 19:58:58.297719 5313 raft_consensus.cc:2243] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.297686 5117 ts_tablet_manager.cc:1916] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.298194 5313 raft_consensus.cc:2272] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.300046 5313 ts_tablet_manager.cc:1916] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.301959 4818 tablet_bootstrap.cc:492] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:58.305378 4513 tablet_service.cc:1511] Processing CreateTablet for tablet b5622394d56142be9072cd1fefc9d48a (DEFAULT_TABLE table=default.Table_a86a2cb6fb394ec19c8ef6599fdfff22 [id=f501210ed93f4890985ec339c43fd11f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.306335 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5622394d56142be9072cd1fefc9d48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.308513 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 537524471691479198beb873a64ee3ae (DEFAULT_TABLE table=default.Table_df7f2cb342be4e7aaab6f206bb9e3c0b [id=c82a65dc130e45b7a33cb292b4a5f088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.308962 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 5d93cb1b53dd4681a7b1b90a2d6b6bdd (DEFAULT_TABLE table=default.Table_79f86b4f9880405db20eade1a857c8e8 [id=2385c0800a004ed8bd6ffc2e8db368f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.309551 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 537524471691479198beb873a64ee3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.309926 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d93cb1b53dd4681a7b1b90a2d6b6bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.310976 5303 raft_consensus.cc:493] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:58.311071 4356 tablet_service.cc:1511] Processing CreateTablet for tablet b5622394d56142be9072cd1fefc9d48a (DEFAULT_TABLE table=default.Table_a86a2cb6fb394ec19c8ef6599fdfff22 [id=f501210ed93f4890985ec339c43fd11f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.311367 5303 raft_consensus.cc:515] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.311933 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5622394d56142be9072cd1fefc9d48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.313539 5303 leader_election.cc:290] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:58.315101 4380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ba03500c0540cda44debe362278e47" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:58.315668 4380 raft_consensus.cc:2468] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 0.
I20260430 19:58:58.316874 4470 leader_election.cc:304] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:58.317984 5266 raft_consensus.cc:2804] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:58.318305 5266 raft_consensus.cc:493] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:58.318648 5266 raft_consensus.cc:3060] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:58:58.297168 4024 rpcz_store.cc:269] 0430 19:58:57.528399 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:57.528654 (+ 255us) service_pool.cc:225] Handling call
0430 19:58:57.588956 (+ 60302us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b5400030554. stack: 00007f2920219c83 00007f2920219b80 00007f291fd8a574 000000000050e094 000000000050e029 00007f2927269470 00007f292730831a 00007f292901c46c 00007f29291163ee 00007f2926d1d4a4 00007f2926d1d432 00007f2926d1d361 00007f2926d1d2dc 00007f2926d1c592 00007f2924ff30ac 00007f2924ff24e6
0430 19:58:58.294021 (+705065us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":132538752}
I20260430 19:58:58.326278 4498 tablet_service.cc:1558] Processing DeleteTablet for tablet 3423379d1e3d4020a4241bf8917f1050 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f0489b4631346f4952ef648e0653242) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:58.328061 4504 tablet_service.cc:1558] Processing DeleteTablet for tablet 443308f2a26f4622aa4551ff5f73fc39 with delete_type TABLET_DATA_DELETED (replaced by tablet 70e4d5af75d04f6d9091d2b086e4a6ba) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:58.328754 4498 tablet_service.cc:1558] Processing DeleteTablet for tablet b6acec54bc7b4692894333e0150a1aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:48 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:58.333351 5266 raft_consensus.cc:515] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.339196 4729 tablet_bootstrap.cc:492] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:58.339671 4729 ts_tablet_manager.cc:1403] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.274s user 0.010s sys 0.000s
I20260430 19:58:58.342072 4729 raft_consensus.cc:359] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.359522 4729 raft_consensus.cc:385] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.359892 4729 raft_consensus.cc:740] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.360376 4028 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 3452fa52c20142f7bf396bc182a829a7 (table default.Table_f4523dfb8ab9414bad907abc56460874 [id=69dac89c4d4a4081aed4ac000a4a51c5]). Expected version 1 got 0
I20260430 19:58:58.361013 4729 consensus_queue.cc:260] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.364565 4356 tablet_service.cc:1511] Processing CreateTablet for tablet 537524471691479198beb873a64ee3ae (DEFAULT_TABLE table=default.Table_df7f2cb342be4e7aaab6f206bb9e3c0b [id=c82a65dc130e45b7a33cb292b4a5f088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.364810 4729 ts_tablet_manager.cc:1434] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.025s user 0.002s sys 0.002s
I20260430 19:58:58.365366 4380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ba03500c0540cda44debe362278e47" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e"
I20260430 19:58:58.365634 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 537524471691479198beb873a64ee3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.365855 4380 raft_consensus.cc:3060] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:58.368810 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 80c14ba8ccc5473192d6070c5cf93684 (DEFAULT_TABLE table=default.Table_aa816c09c98b4870b953e6da12736c04 [id=d7231ca32ba641bb9367f6d60261f4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.372982 4357 tablet_service.cc:1558] Processing DeleteTablet for tablet d1f7489f1d9c46b8b33745c3e03a2138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.373431 5313 ts_tablet_manager.cc:1929] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.373802 5313 log.cc:1199] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/395f97fdf3834cb9aaabd261e1bf76c5
I20260430 19:58:58.374243 4729 tablet_bootstrap.cc:492] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:58.376367 4380 raft_consensus.cc:2468] T 28ba03500c0540cda44debe362278e47 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a11a9a7f244435b8e84554e3fabbd0f in term 1.
I20260430 19:58:58.381349 4357 tablet_service.cc:1511] Processing CreateTablet for tablet 80c14ba8ccc5473192d6070c5cf93684 (DEFAULT_TABLE table=default.Table_aa816c09c98b4870b953e6da12736c04 [id=d7231ca32ba641bb9367f6d60261f4e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.381876 4818 tablet_bootstrap.cc:654] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.382488 4357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c14ba8ccc5473192d6070c5cf93684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.377345 4470 leader_election.cc:304] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:58.387643 5303 raft_consensus.cc:2804] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:58:58.378722 5266 leader_election.cc:290] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: Requested vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:58.386801 4513 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ba03500c0540cda44debe362278e47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:49 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:58.374505 5313 ts_tablet_manager.cc:1950] T 395f97fdf3834cb9aaabd261e1bf76c5 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:58.405682 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 395f97fdf3834cb9aaabd261e1bf76c5 (table default.Table_cb48b45e8d654862861b868ea8fd0368 [id=49febbf4c6a74d18bc4e1d495d1f8890]) successfully deleted
I20260430 19:58:58.405633 4513 tablet_service.cc:1511] Processing CreateTablet for tablet daa53112b685499e83d9add4f4b67f5b (DEFAULT_TABLE table=default.Table_7e85838feea147dca5c3b2e154d654b4 [id=00c12c7f9e2c4601b99118fb2a0208c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.406102 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c14ba8ccc5473192d6070c5cf93684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.409153 4818 tablet_bootstrap.cc:492] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:58.409576 4818 ts_tablet_manager.cc:1403] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.108s user 0.008s sys 0.000s
I20260430 19:58:58.410130 4361 tablet_service.cc:1511] Processing CreateTablet for tablet daa53112b685499e83d9add4f4b67f5b (DEFAULT_TABLE table=default.Table_7e85838feea147dca5c3b2e154d654b4 [id=00c12c7f9e2c4601b99118fb2a0208c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.411119 4361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa53112b685499e83d9add4f4b67f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.411988 4818 raft_consensus.cc:359] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.412582 4818 raft_consensus.cc:385] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.412806 4818 raft_consensus.cc:740] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.416575 5117 ts_tablet_manager.cc:1929] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.416921 5117 log.cc:1199] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/395f97fdf3834cb9aaabd261e1bf76c5
I20260430 19:58:58.417651 5117 ts_tablet_manager.cc:1950] T 395f97fdf3834cb9aaabd261e1bf76c5 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:58.419701 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 213) took 2513 ms. Trace:
I20260430 19:58:58.419871 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 02fa8d8338bd406e9fa388bf3c1ef729 (DEFAULT_TABLE table=default.Table_3b4274cea2394743828eb6d1a31fdbbe [id=fe2351bafa2a45adbf3704df8225ab81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.419994 5117 rpcz_store.cc:276] 0430 19:58:55.906640 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:55.906799 (+ 159us) service_pool.cc:225] Handling call
0430 19:58:58.284193 (+2377394us) ts_tablet_manager.cc:1151] Deleting tablet 395f97fdf3834cb9aaabd261e1bf76c5
0430 19:58:58.284246 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.361734 (+ 77488us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.416463 (+ 54729us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:58.419648 (+ 3185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5780,"tablet-delete.queue_time_us":2376813}
I20260430 19:58:58.413434 4818 consensus_queue.cc:260] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.421118 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02fa8d8338bd406e9fa388bf3c1ef729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.421535 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 395f97fdf3834cb9aaabd261e1bf76c5 (table default.Table_cb48b45e8d654862861b868ea8fd0368 [id=49febbf4c6a74d18bc4e1d495d1f8890]) successfully deleted
I20260430 19:58:58.434041 4818 ts_tablet_manager.cc:1434] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260430 19:58:58.434006 4510 tablet_service.cc:1511] Processing CreateTablet for tablet 02fa8d8338bd406e9fa388bf3c1ef729 (DEFAULT_TABLE table=default.Table_3b4274cea2394743828eb6d1a31fdbbe [id=fe2351bafa2a45adbf3704df8225ab81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.443125 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02fa8d8338bd406e9fa388bf3c1ef729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.444466 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daa53112b685499e83d9add4f4b67f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.433957 5313 tablet_replica.cc:333] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:58.448652 4818 tablet_bootstrap.cc:492] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:58.452487 5313 raft_consensus.cc:2243] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.452951 5313 raft_consensus.cc:2272] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.454722 5313 ts_tablet_manager.cc:1916] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.456954 4509 tablet_service.cc:1511] Processing CreateTablet for tablet 956bfdee8cbf46db9909d991160565a7 (DEFAULT_TABLE table=default.Table_bf80dfa8947243c69b46cbac49bfe41a [id=46b84aeb59b84ebda5bc4be50e361e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.458057 4509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956bfdee8cbf46db9909d991160565a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.461565 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 221) took 1667 ms. Trace:
I20260430 19:58:58.461882 5117 rpcz_store.cc:276] 0430 19:58:56.793969 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:56.794088 (+ 119us) service_pool.cc:225] Handling call
0430 19:58:58.461153 (+1667065us) ts_tablet_manager.cc:1151] Deleting tablet 958cf4732e254c75bda36a6a550b93ef
0430 19:58:58.461210 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.461494 (+ 284us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1666553}
I20260430 19:58:58.463094 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 222) took 1668 ms. Trace:
I20260430 19:58:58.463497 4729 tablet_bootstrap.cc:654] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.463378 5117 rpcz_store.cc:276] 0430 19:58:56.794776 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:56.794889 (+ 113us) service_pool.cc:225] Handling call
0430 19:58:58.462713 (+1667824us) ts_tablet_manager.cc:1151] Deleting tablet ebd991541273429e9c2bdc018ee402f5
0430 19:58:58.462767 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.463039 (+ 272us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1667300}
I20260430 19:58:58.464484 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 220) took 1671 ms. Trace:
I20260430 19:58:58.464757 5117 rpcz_store.cc:276] 0430 19:58:56.793145 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:56.801578 (+ 8433us) service_pool.cc:225] Handling call
0430 19:58:58.464127 (+1662549us) ts_tablet_manager.cc:1151] Deleting tablet 239a2c6a534f4f478e436d14f94cd762
0430 19:58:58.464183 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.464439 (+ 256us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1627176}
I20260430 19:58:58.465370 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet f5c54c37bdae458d88594a3f5c401a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53688
W20260430 19:58:58.465960 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 958cf4732e254c75bda36a6a550b93ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 958cf4732e254c75bda36a6a550b93ef already in progress: creating tablet
I20260430 19:58:58.466624 4818 tablet_bootstrap.cc:654] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:58.466614 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet ebd991541273429e9c2bdc018ee402f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebd991541273429e9c2bdc018ee402f5 already in progress: creating tablet
W20260430 19:58:58.467164 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet 239a2c6a534f4f478e436d14f94cd762 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 239a2c6a534f4f478e436d14f94cd762 already in progress: creating tablet
I20260430 19:58:58.471428 5266 raft_consensus.cc:493] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:58.472376 5266 raft_consensus.cc:515] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.478049 5266 leader_election.cc:290] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:58.479874 4380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4cf90db44540f2b438966361af8192" candidate_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" is_pre_election: true
I20260430 19:58:58.480667 4380 raft_consensus.cc:2393] T ba4cf90db44540f2b438966361af8192 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a11a9a7f244435b8e84554e3fabbd0f in current term 1: Already voted for candidate 5a7356e99e744525861e12c108cfa3fe in this term.
I20260430 19:58:58.486913 5303 raft_consensus.cc:697] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Becoming Leader. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Running, Role: LEADER
I20260430 19:58:58.487845 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 226) took 1290 ms. Trace:
I20260430 19:58:58.488188 5117 rpcz_store.cc:276] 0430 19:58:57.197496 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:57.197603 (+ 107us) service_pool.cc:225] Handling call
0430 19:58:58.465457 (+1267854us) ts_tablet_manager.cc:1151] Deleting tablet d703e0b6c6cb433d9c49b5d7956ae360
0430 19:58:58.465513 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.487787 (+ 22274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328,"tablet-delete.queue_time_us":1198590}
W20260430 19:58:58.489344 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet d703e0b6c6cb433d9c49b5d7956ae360 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d703e0b6c6cb433d9c49b5d7956ae360 already in progress: creating tablet
I20260430 19:58:58.490001 5117 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53688 (request call id 227) took 1291 ms. Trace:
I20260430 19:58:58.490226 4504 tablet_service.cc:1558] Processing DeleteTablet for tablet ba4cf90db44540f2b438966361af8192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:58.490303 5117 rpcz_store.cc:276] 0430 19:58:57.198485 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:57.198621 (+ 136us) service_pool.cc:225] Handling call
0430 19:58:58.489629 (+1291008us) ts_tablet_manager.cc:1151] Deleting tablet d846bbe4af0544e88ea5012ccc815881
0430 19:58:58.489681 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:58.489956 (+ 275us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1222530}
I20260430 19:58:58.491629 5117 tablet_replica.cc:333] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:58.492380 5117 raft_consensus.cc:2243] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:58.493288 5117 raft_consensus.cc:2272] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260430 19:58:58.494206 4016 catalog_manager.cc:4982] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet d846bbe4af0544e88ea5012ccc815881 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d846bbe4af0544e88ea5012ccc815881 already in progress: creating tablet
I20260430 19:58:58.493461 5303 consensus_queue.cc:237] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.498332 5174 raft_consensus.cc:493] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:58.498783 5174 raft_consensus.cc:515] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.499912 5117 ts_tablet_manager.cc:1916] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.500586 5174 leader_election.cc:290] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:58.507520 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 956bfdee8cbf46db9909d991160565a7 (DEFAULT_TABLE table=default.Table_bf80dfa8947243c69b46cbac49bfe41a [id=46b84aeb59b84ebda5bc4be50e361e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.508692 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956bfdee8cbf46db9909d991160565a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.511482 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5c54c37bdae458d88594a3f5c401a48" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
W20260430 19:58:58.513007 4317 leader_election.cc:343] T f5c54c37bdae458d88594a3f5c401a48 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:58.521467 4504 tablet_service.cc:1511] Processing CreateTablet for tablet e258d354393e42639e9ae519e8961a6a (DEFAULT_TABLE table=default.Table_6b7e49e3e1ee4ce4bfa8075c27cba432 [id=67ecb34de7af400a80b3936cf082cf83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.522399 4729 tablet_bootstrap.cc:492] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:58.522816 4729 ts_tablet_manager.cc:1403] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.157s user 0.011s sys 0.000s
I20260430 19:58:58.527396 4818 tablet_bootstrap.cc:492] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:58.527799 4818 ts_tablet_manager.cc:1403] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.079s user 0.024s sys 0.005s
I20260430 19:58:58.530128 4818 raft_consensus.cc:359] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.530735 4818 raft_consensus.cc:385] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.530975 4818 raft_consensus.cc:740] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.531122 4504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e258d354393e42639e9ae519e8961a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.531504 4818 consensus_queue.cc:260] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.533471 4818 ts_tablet_manager.cc:1434] T 66e94750de27416e9adcf60fccf2ace9 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260430 19:58:58.545074 4818 tablet_bootstrap.cc:492] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:58.547605 4729 raft_consensus.cc:359] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.548277 4729 raft_consensus.cc:385] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.548516 4729 raft_consensus.cc:740] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.549124 4729 consensus_queue.cc:260] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.551358 4818 tablet_bootstrap.cc:654] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.553651 4021 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 66e94750de27416e9adcf60fccf2ace9 (table default.Table_63f9f99010054a52adde13533fc41269 [id=245a6d903af145969e0ff7d40eb5f1e7]). Expected version 2 got 0
I20260430 19:58:58.557082 4729 ts_tablet_manager.cc:1434] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.034s user 0.005s sys 0.000s
I20260430 19:58:58.557050 4356 tablet_service.cc:1511] Processing CreateTablet for tablet e258d354393e42639e9ae519e8961a6a (DEFAULT_TABLE table=default.Table_6b7e49e3e1ee4ce4bfa8075c27cba432 [id=67ecb34de7af400a80b3936cf082cf83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.562553 4356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e258d354393e42639e9ae519e8961a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.562486 4498 tablet_service.cc:1511] Processing CreateTablet for tablet 95b78965508247cface36a104d1ead9f (DEFAULT_TABLE table=default.Table_2e10c9590ede4aca976afa6a1e397c7b [id=f772cd158c34469ca0ee7fe4013c9365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.563527 4498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b78965508247cface36a104d1ead9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.568568 4818 tablet_bootstrap.cc:492] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:58.569072 4818 ts_tablet_manager.cc:1403] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.001s
I20260430 19:58:58.571396 4818 raft_consensus.cc:359] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.572014 4818 raft_consensus.cc:385] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.572261 4818 raft_consensus.cc:740] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.572788 4818 consensus_queue.cc:260] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.574684 4818 ts_tablet_manager.cc:1434] T 6f39fb64a301447f988c53ccfe896074 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260430 19:58:58.575006 5313 ts_tablet_manager.cc:1929] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:58.575556 4818 tablet_bootstrap.cc:492] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:58.597011 4818 tablet_bootstrap.cc:654] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.600739 5313 log.cc:1199] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/db0efaecfef44457a5128e878796c061
I20260430 19:58:58.601531 5313 ts_tablet_manager.cc:1950] T db0efaecfef44457a5128e878796c061 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:58.604681 4818 tablet_bootstrap.cc:492] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:58.605060 4818 ts_tablet_manager.cc:1403] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.008s
I20260430 19:58:58.607355 4818 raft_consensus.cc:359] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.607919 4818 raft_consensus.cc:385] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.608174 4818 raft_consensus.cc:740] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.594684 4729 tablet_bootstrap.cc:492] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:58.608711 4818 consensus_queue.cc:260] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.617715 4729 tablet_bootstrap.cc:654] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.619449 4021 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 66e94750de27416e9adcf60fccf2ace9 (table default.Table_63f9f99010054a52adde13533fc41269 [id=245a6d903af145969e0ff7d40eb5f1e7]). Expected version 2 got 0
I20260430 19:58:58.623126 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 95b78965508247cface36a104d1ead9f (DEFAULT_TABLE table=default.Table_2e10c9590ede4aca976afa6a1e397c7b [id=f772cd158c34469ca0ee7fe4013c9365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.624501 4363 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f39fb64a301447f988c53ccfe896074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:52 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.627513 5333 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:58.627969 5333 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:58.629750 5333 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:58.630690 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:58.631836 3957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:58.632668 4363 tablet_service.cc:1511] Processing CreateTablet for tablet 4c2adbc9008d4397ade546b1b3f59f13 (DEFAULT_TABLE table=default.Table_79c910b252344461a992f1d1dc1bd625 [id=6217bfcecf2c418c96989c41db28e1da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.633632 4818 ts_tablet_manager.cc:1434] T 4758b3d846434b2b8466033c083b7339 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260430 19:58:58.652369 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c2adbc9008d4397ade546b1b3f59f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.652695 4818 tablet_bootstrap.cc:492] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:58.623270 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 4c2adbc9008d4397ade546b1b3f59f13 (DEFAULT_TABLE table=default.Table_79c910b252344461a992f1d1dc1bd625 [id=6217bfcecf2c418c96989c41db28e1da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.651466 4954 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:58.655306 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c2adbc9008d4397ade546b1b3f59f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.657724 4818 tablet_bootstrap.cc:654] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.637984 5303 consensus_queue.cc:1048] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:58:58.665380 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b78965508247cface36a104d1ead9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.670763 5333 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:58:58.672207 5117 ts_tablet_manager.cc:1929] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.672539 5117 log.cc:1199] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/3423379d1e3d4020a4241bf8917f1050
I20260430 19:58:58.673173 5117 ts_tablet_manager.cc:1950] T 3423379d1e3d4020a4241bf8917f1050 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:58.674620 4509 tablet_service.cc:1558] Processing DeleteTablet for tablet f5c54c37bdae458d88594a3f5c401a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53008
I20260430 19:58:58.679097 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 3423379d1e3d4020a4241bf8917f1050 (table default.Table_60d3a33c8bfc4641ac60f0ea151c1225 [id=37e26a1d9e4f40cc8fb14b9934ba3780]) successfully deleted
I20260430 19:58:58.683043 4818 tablet_bootstrap.cc:492] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e: No bootstrap required, opened a new log
I20260430 19:58:58.683444 4818 ts_tablet_manager.cc:1403] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.001s
W20260430 19:58:58.684103 4016 catalog_manager.cc:4969] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): delete failed for tablet db0efaecfef44457a5128e878796c061 because the tablet was not found. No further retry: Not found: Tablet not found: db0efaecfef44457a5128e878796c061
I20260430 19:58:58.685691 5313 tablet_replica.cc:333] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:58.646674 4504 tablet_service.cc:1511] Processing CreateTablet for tablet baf0eede482644189537ea3da24936da (DEFAULT_TABLE table=default.Table_5ff3da24c8fe4a829ccbedd744bfb835 [id=aa3f732ab9f0466ca91db6ffac18ce09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.686488 5313 raft_consensus.cc:2243] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:58.685056 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet db0efaecfef44457a5128e878796c061 (table default.Table_6f2bc72f8d7946e78d1e9c7ca4de1a6c [id=d4bfa57fdfc64108ba1e909d0230fb51]) successfully deleted
I20260430 19:58:58.686722 4504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baf0eede482644189537ea3da24936da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.688064 5313 raft_consensus.cc:2272] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.692022 5313 ts_tablet_manager.cc:1916] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.695600 4818 raft_consensus.cc:359] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.696244 4818 raft_consensus.cc:385] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.696491 4818 raft_consensus.cc:740] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.688398 5117 tablet_replica.cc:333] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:58.696719 4729 tablet_bootstrap.cc:492] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:58.697103 4729 ts_tablet_manager.cc:1403] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.103s user 0.008s sys 0.000s
I20260430 19:58:58.697031 4818 consensus_queue.cc:260] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.699054 4818 ts_tablet_manager.cc:1434] T 5563cfa73f194db69a6978d348f1c073 P d8345ec7f57c4399a8a4afe4dd43439e: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260430 19:58:58.699395 4729 raft_consensus.cc:359] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.699900 4818 tablet_bootstrap.cc:492] T bdc476c6909d4c998f8b102029eb62a4 P d8345ec7f57c4399a8a4afe4dd43439e: Bootstrap starting.
I20260430 19:58:58.699952 4729 raft_consensus.cc:385] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.700209 4729 raft_consensus.cc:740] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.709954 5117 raft_consensus.cc:2243] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.710479 5117 raft_consensus.cc:2272] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.711129 4729 consensus_queue.cc:260] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.716820 5117 ts_tablet_manager.cc:1916] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.721535 4818 tablet_bootstrap.cc:654] T bdc476c6909d4c998f8b102029eb62a4 P d8345ec7f57c4399a8a4afe4dd43439e: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.727864 5174 raft_consensus.cc:493] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:58.728353 5174 raft_consensus.cc:515] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.729915 4510 tablet_service.cc:1511] Processing CreateTablet for tablet 78151650b58345d28127ae16c8a58c0c (DEFAULT_TABLE table=default.Table_d35337f4d43a45e48e14b9f523681d04 [id=b3840f67482c4b668da7ea4005543d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.730134 5174 leader_election.cc:290] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:58.732933 4729 ts_tablet_manager.cc:1434] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260430 19:58:58.739777 4363 tablet_service.cc:1511] Processing CreateTablet for tablet baf0eede482644189537ea3da24936da (DEFAULT_TABLE table=default.Table_5ff3da24c8fe4a829ccbedd744bfb835 [id=aa3f732ab9f0466ca91db6ffac18ce09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.741596 4363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baf0eede482644189537ea3da24936da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.742952 5174 raft_consensus.cc:493] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:58.746814 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ba8046c294be3ac20e2d93925d17b" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:58.747285 4729 tablet_bootstrap.cc:492] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:58.747433 5313 ts_tablet_manager.cc:1929] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.743381 5174 raft_consensus.cc:515] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.748242 4356 tablet_service.cc:1558] Processing DeleteTablet for tablet 4758b3d846434b2b8466033c083b7339 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:53 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.749734 5174 leader_election.cc:290] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953)
I20260430 19:58:58.747411 4532 raft_consensus.cc:2468] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 0.
I20260430 19:58:58.750278 4021 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 958ba8046c294be3ac20e2d93925d17b (table default.Table_7b77d87732e142a7b0687c71ce62d0c3 [id=ffb188d244694ce2aa7985340abb8813]). Expected version 1 got 0
I20260430 19:58:58.751114 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3452fa52c20142f7bf396bc182a829a7" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" is_pre_election: true
I20260430 19:58:58.751655 4317 leader_election.cc:304] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:58.750813 5313 log.cc:1199] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/081a060c44924bcda09ffc8fe88012b3
W20260430 19:58:58.752511 4317 leader_election.cc:343] T 3452fa52c20142f7bf396bc182a829a7 P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:58:58.753062 5174 raft_consensus.cc:2804] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:58:58.753405 5174 raft_consensus.cc:493] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:58:58.753712 5174 raft_consensus.cc:3060] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:58.754805 5313 ts_tablet_manager.cc:1950] T 081a060c44924bcda09ffc8fe88012b3 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:58.757681 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 081a060c44924bcda09ffc8fe88012b3 (table default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]) successfully deleted
I20260430 19:58:58.731438 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78151650b58345d28127ae16c8a58c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.768568 4729 tablet_bootstrap.cc:654] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:58.772100 5174 raft_consensus.cc:515] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.773907 5174 leader_election.cc:290] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Requested vote from peers 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953), 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065)
I20260430 19:58:58.774940 4532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ba8046c294be3ac20e2d93925d17b" candidate_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a11a9a7f244435b8e84554e3fabbd0f"
I20260430 19:58:58.775470 4532 raft_consensus.cc:3060] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:58.779464 4021 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 958ba8046c294be3ac20e2d93925d17b (table default.Table_7b77d87732e142a7b0687c71ce62d0c3 [id=ffb188d244694ce2aa7985340abb8813]). Expected version 1 got 0
I20260430 19:58:58.782816 4532 raft_consensus.cc:2468] T 958ba8046c294be3ac20e2d93925d17b P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8345ec7f57c4399a8a4afe4dd43439e in term 1.
W20260430 19:58:58.784034 4013 catalog_manager.cc:4982] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): delete failed for tablet 619fb8dd24244d1ea81953668659df8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 619fb8dd24244d1ea81953668659df8a already in progress: creating tablet
I20260430 19:58:58.784358 4317 leader_election.cc:304] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f, d8345ec7f57c4399a8a4afe4dd43439e; no voters:
I20260430 19:58:58.785665 5313 tablet_replica.cc:333] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:58.786535 5313 raft_consensus.cc:2243] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.787614 5117 ts_tablet_manager.cc:1929] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.787985 5117 log.cc:1199] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/081a060c44924bcda09ffc8fe88012b3
I20260430 19:58:58.788388 5174 raft_consensus.cc:2804] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Leader election won for term 1
W20260430 19:58:58.789223 4014 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet cfd78a1cd7a94e27bb376d1b7c151073 (table default.Table_d1543ae16be840548845a7781aa09c80 [id=28b54770e4484fb28a9ceb33742ba620], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cfd78a1cd7a94e27bb376d1b7c151073 consensus state has no leader
I20260430 19:58:58.798976 5313 raft_consensus.cc:2272] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.802618 5174 raft_consensus.cc:697] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [term 1 LEADER]: Becoming Leader. State: Replica: d8345ec7f57c4399a8a4afe4dd43439e, State: Running, Role: LEADER
I20260430 19:58:58.803884 5174 consensus_queue.cc:237] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:58.804385 5313 ts_tablet_manager.cc:1916] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.788692 5117 ts_tablet_manager.cc:1950] T 081a060c44924bcda09ffc8fe88012b3 P d8345ec7f57c4399a8a4afe4dd43439e: Deleting consensus metadata
I20260430 19:58:58.813421 5117 tablet_replica.cc:333] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: stopping tablet replica
I20260430 19:58:58.816071 4016 catalog_manager.cc:5002] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): tablet 081a060c44924bcda09ffc8fe88012b3 (table default.Table_733dfaaf74774e8db1bfa300e4e56ced [id=49546fb9471b4a1c969f4b438ec6ef3a]) successfully deleted
I20260430 19:58:58.817513 4351 tablet_service.cc:1558] Processing DeleteTablet for tablet 5563cfa73f194db69a6978d348f1c073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-04-30 19:58:50 UTC) from {username='slave'} at 127.0.0.1:53688
I20260430 19:58:58.818620 5117 raft_consensus.cc:2243] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.820005 5117 pending_rounds.cc:70] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: Trying to abort 1 pending ops.
I20260430 19:58:58.821076 5117 pending_rounds.cc:77] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964140296929280 op_type: NO_OP noop_request { }
I20260430 19:58:58.822358 5117 raft_consensus.cc:2889] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:58.823566 5117 raft_consensus.cc:2272] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.825858 5344 tablet.cc:1724] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260430 19:58:58.827909 5117 ts_tablet_manager.cc:1916] T 70d410f3b3b04e649b79dda04c0891ee P d8345ec7f57c4399a8a4afe4dd43439e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.834803 4351 tablet_service.cc:1511] Processing CreateTablet for tablet 78151650b58345d28127ae16c8a58c0c (DEFAULT_TABLE table=default.Table_d35337f4d43a45e48e14b9f523681d04 [id=b3840f67482c4b668da7ea4005543d0f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:58.835079 4320 connection.cc:570] client connection to 127.3.190.1:33065 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:58.835501 4320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260430 19:58:58.835831 4351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78151650b58345d28127ae16c8a58c0c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:58.836995 4470 connection.cc:570] client connection to 127.3.190.1:33065 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:58.837390 4470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:58.851032 4013 connection.cc:570] server connection from 127.3.190.1:59387 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260430 19:58:58.862748 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 4280
W20260430 19:58:58.870307 4016 connection.cc:570] client connection to 127.3.190.1:33065 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:58.870818 4016 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 1 similar messages]
W20260430 19:58:58.875761 5350 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:58:58.849925 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.1:33065 (local address 127.3.190.3:45005)
0430 19:58:58.870957 (+ 21032us) negotiation.cc:107] Waiting for socket to connect
0430 19:58:58.871245 (+ 288us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":18948,"thread_start_us":19287,"threads_started":1}
W20260430 19:58:58.886904 4470 consensus_peers.cc:597] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:58.887317 4470 consensus_peers.cc:597] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:58.887640 4498 tablet_service.cc:1511] Processing CreateTablet for tablet c5cceaf2291441b4b0395bc31104417c (DEFAULT_TABLE table=default.Table_9e269db71b714e8084c34773f70acf2e [id=6f0b2637765741cfb9bc76ef59de94f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.888715 4498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5cceaf2291441b4b0395bc31104417c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.878563 5313 ts_tablet_manager.cc:1929] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:58:58.880332 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 3e002c260ea249d9b2572b353732a847 (DEFAULT_TABLE table=default.Table_c507e6b6dbb2496e90f0ec6eab3a02c6 [id=e095590aa1d84d688fee8946c36387a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.883453 4513 tablet_service.cc:1511] Processing CreateTablet for tablet f85210bc3dab48bfb6d93f53910d6062 (DEFAULT_TABLE table=default.Table_2d7bda6327c844a2940c048b9bf2ff56 [id=2931d2ff44544865aa97fa221dbd3ba6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:58:58.887667 4470 leader_election.cc:336] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104)
I20260430 19:58:58.891490 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e002c260ea249d9b2572b353732a847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.895203 4513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f85210bc3dab48bfb6d93f53910d6062. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:58.891499 4470 leader_election.cc:336] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104)
W20260430 19:58:58.899480 4470 consensus_peers.cc:597] T c78d50690da7428fadb848dadefec47c P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:58.899839 4470 consensus_peers.cc:597] T ed157967cdad480f88963a59fe85b54b P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:58.900180 4470 leader_election.cc:336] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104)
I20260430 19:58:58.900506 4470 leader_election.cc:304] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:58.901180 5303 raft_consensus.cc:3060] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:58:58.906828 4729 tablet_bootstrap.cc:492] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:58.907228 5313 log.cc:1199] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/3423379d1e3d4020a4241bf8917f1050
I20260430 19:58:58.908030 5313 ts_tablet_manager.cc:1950] T 3423379d1e3d4020a4241bf8917f1050 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:58.908306 5303 raft_consensus.cc:2749] T ba4cf90db44540f2b438966361af8192 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:58:58.907241 4729 ts_tablet_manager.cc:1403] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.160s user 0.014s sys 0.010s
I20260430 19:58:58.910789 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 3423379d1e3d4020a4241bf8917f1050 (table default.Table_60d3a33c8bfc4641ac60f0ea151c1225 [id=37e26a1d9e4f40cc8fb14b9934ba3780]) successfully deleted
I20260430 19:58:58.911854 4729 raft_consensus.cc:359] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.919047 4729 raft_consensus.cc:385] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.919322 4729 raft_consensus.cc:740] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.922238 4729 consensus_queue.cc:260] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.924125 4729 ts_tablet_manager.cc:1434] T ce21ac617c5b4d6ca97c6f68f53c6a67 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260430 19:58:58.939842 5313 tablet_replica.cc:333] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:58.940689 5313 raft_consensus.cc:2243] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:58:58.941177 5313 raft_consensus.cc:2272] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:58.943342 5313 ts_tablet_manager.cc:1916] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:58.943831 4729 tablet_bootstrap.cc:492] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:58.950464 4729 tablet_bootstrap.cc:654] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:58.958133 4470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 57957.6us
W20260430 19:58:58.958674 4016 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.190.1:33065, user_credentials={real_user=slave}} blocked reactor thread for 87855.6us
W20260430 19:58:58.966763 4470 consensus_peers.cc:597] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:58.967417 4470 consensus_peers.cc:597] T edb14746d8564bf8833e933a01771492 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:58.878521 4021 catalog_manager.cc:5671] T 958ba8046c294be3ac20e2d93925d17b P d8345ec7f57c4399a8a4afe4dd43439e reported cstate change: term changed from 0 to 1, leader changed from <none> to d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2). New cstate: current_term: 1 leader_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } health_report { overall_health: HEALTHY } } }
I20260430 19:58:58.969883 4021 catalog_manager.cc:5763] TS d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939) does not have the latest schema for tablet 958ba8046c294be3ac20e2d93925d17b (table default.Table_7b77d87732e142a7b0687c71ce62d0c3 [id=ffb188d244694ce2aa7985340abb8813]). Expected version 1 got 0
I20260430 19:58:58.978185 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 7e9791626eae43b684ebcb86c7cf32fa (DEFAULT_TABLE table=default.Table_efddd98fead448ea9245be8d1f3ccbbd [id=7ade4134362b438ea5d3037db711f710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:58.979344 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e9791626eae43b684ebcb86c7cf32fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:58.988270 4729 tablet_bootstrap.cc:492] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:58.988675 4729 ts_tablet_manager.cc:1403] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.045s user 0.010s sys 0.000s
I20260430 19:58:58.994129 4729 raft_consensus.cc:359] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.994747 4729 raft_consensus.cc:385] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:58.994984 4729 raft_consensus.cc:740] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:58.995512 4729 consensus_queue.cc:260] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:58.997437 4729 ts_tablet_manager.cc:1434] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260430 19:58:59.005684 4729 tablet_bootstrap.cc:492] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:59.010756 4729 tablet_bootstrap.cc:654] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:59.035465 4729 tablet_bootstrap.cc:492] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:59.035884 4729 ts_tablet_manager.cc:1403] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.003s
I20260430 19:58:59.038338 4729 raft_consensus.cc:359] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.038949 4729 raft_consensus.cc:385] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:59.039181 4729 raft_consensus.cc:740] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
W20260430 19:58:59.043447 4470 consensus_peers.cc:597] T d4e34f3b1bf84f72855cf87683c5e318 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:59.046003 5313 ts_tablet_manager.cc:1929] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:58:59.046433 5313 log.cc:1199] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/443308f2a26f4622aa4551ff5f73fc39
I20260430 19:58:59.047178 5313 ts_tablet_manager.cc:1950] T 443308f2a26f4622aa4551ff5f73fc39 P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:59.039884 4729 consensus_queue.cc:260] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.052922 4510 tablet_service.cc:1511] Processing CreateTablet for tablet faa69ffb02e445d3a06c95358d38a0e7 (DEFAULT_TABLE table=default.Table_cf9ef037d1634978a0ddbe0a06f32284 [id=567cc975cfeb451087657471b5b225d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:59.068138 4729 ts_tablet_manager.cc:1434] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.032s user 0.003s sys 0.003s
I20260430 19:58:59.070550 4729 tablet_bootstrap.cc:492] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:59.072549 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 671c14f175544423b7ee03733caffe95 (DEFAULT_TABLE table=default.Table_839a70940e6f4a6a96ac5cec16e26a91 [id=e8050d28a7d04cbb9c8241e4a5d7e6b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:58:59.114688 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet 443308f2a26f4622aa4551ff5f73fc39 (table default.Table_74850f27abe54175909dc1d9c494804b [id=7ad67f36177f4ef18a33fa7a65cbef6f]) successfully deleted
I20260430 19:58:59.085137 4729 tablet_bootstrap.cc:654] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:59.093250 4510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet faa69ffb02e445d3a06c95358d38a0e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:58:59.135247 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 671c14f175544423b7ee03733caffe95. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:58:59.139916 4016 catalog_manager.cc:4729] TS 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84b02b29777341d3934f6a77967d79b0: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection refused (error 111)
I20260430 19:58:59.140256 4729 tablet_bootstrap.cc:492] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:59.140659 4729 ts_tablet_manager.cc:1403] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.070s user 0.007s sys 0.009s
I20260430 19:58:59.143137 4729 raft_consensus.cc:359] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.143764 4729 raft_consensus.cc:385] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:59.144025 4729 raft_consensus.cc:740] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:59.144793 4729 consensus_queue.cc:260] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.115733 5313 tablet_replica.cc:333] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:59.146234 5313 raft_consensus.cc:2243] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:59.147141 5313 raft_consensus.cc:2272] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:59.149740 5313 ts_tablet_manager.cc:1916] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:58:59.159701 4729 ts_tablet_manager.cc:1434] T 4758b3d846434b2b8466033c083b7339 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
W20260430 19:58:59.176661 4470 consensus_peers.cc:597] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:59.181422 4729 tablet_bootstrap.cc:492] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:59.193661 5303 raft_consensus.cc:493] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:59.194113 5303 raft_consensus.cc:515] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } }
I20260430 19:58:59.195304 4729 tablet_bootstrap.cc:654] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:59.195953 5303 leader_election.cc:290] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
W20260430 19:58:59.201242 4470 leader_election.cc:336] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection refused (error 111)
W20260430 19:58:59.210989 4470 consensus_peers.cc:597] T 0832f9191b7f4c3683f37cce8966b266 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Couldn't send request to peer 5a7356e99e744525861e12c108cfa3fe. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:58:59.211658 5303 raft_consensus.cc:493] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:59.212328 5303 raft_consensus.cc:515] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.214107 5303 leader_election.cc:290] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:59.220291 4729 tablet_bootstrap.cc:492] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:59.220733 4729 ts_tablet_manager.cc:1403] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.004s
W20260430 19:58:59.221896 4470 leader_election.cc:336] T 3452fa52c20142f7bf396bc182a829a7 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection refused (error 111)
I20260430 19:58:59.223193 4729 raft_consensus.cc:359] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.223892 4729 raft_consensus.cc:385] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:59.224153 4729 raft_consensus.cc:740] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:59.234915 4729 consensus_queue.cc:260] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.236342 4729 ts_tablet_manager.cc:1434] T 6f39fb64a301447f988c53ccfe896074 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260430 19:58:59.237556 4729 tablet_bootstrap.cc:492] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
I20260430 19:58:59.252298 4729 tablet_bootstrap.cc:654] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
I20260430 19:58:59.299868 5303 raft_consensus.cc:493] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:58:59.302389 5303 raft_consensus.cc:515] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.305799 4028 catalog_manager.cc:6216] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755: default.Table_425e3062d49141638642c50ca47fd102 [id=d963a034463343989c1d699cce2134a5] alter complete (version 1)
I20260430 19:58:59.310503 5303 leader_election.cc:290] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065), d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939)
I20260430 19:58:59.323464 5313 ts_tablet_manager.cc:1929] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260430 19:58:59.324132 4470 leader_election.cc:336] T d1f7489f1d9c46b8b33745c3e03a2138 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 5a7356e99e744525861e12c108cfa3fe (127.3.190.1:33065): Network error: Client connection negotiation failed: client connection to 127.3.190.1:33065: connect: Connection refused (error 111)
I20260430 19:58:59.327842 4729 tablet_bootstrap.cc:492] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:59.328438 4729 ts_tablet_manager.cc:1403] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.002s
I20260430 19:58:59.331637 5364 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbc6c3c322c24063ad0cf6b243cf4755)
I20260430 19:58:59.331857 4729 raft_consensus.cc:359] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.332479 4729 raft_consensus.cc:385] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:58:59.332738 4729 raft_consensus.cc:740] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a11a9a7f244435b8e84554e3fabbd0f, State: Initialized, Role: FOLLOWER
I20260430 19:58:59.333441 4729 consensus_queue.cc:260] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8345ec7f57c4399a8a4afe4dd43439e" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 43939 } } peers { permanent_uuid: "5a7356e99e744525861e12c108cfa3fe" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33065 } } peers { permanent_uuid: "8a11a9a7f244435b8e84554e3fabbd0f" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 37953 } }
I20260430 19:58:59.335047 5364 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b0a440dabed4814aaefe431944a7b5e" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 33781 } } peers { permanent_uuid: "caa5357d0506482ea383b464e24f2a7a" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35559 } } peers { permanent_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 42577 } }
I20260430 19:58:59.338687 4729 ts_tablet_manager.cc:1434] T 66e94750de27416e9adcf60fccf2ace9 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent starting tablet: real 0.010s user 0.004s sys 0.005s
I20260430 19:58:59.324158 5313 log.cc:1199] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/b6acec54bc7b4692894333e0150a1aff
I20260430 19:58:59.339870 5313 ts_tablet_manager.cc:1950] T b6acec54bc7b4692894333e0150a1aff P 8a11a9a7f244435b8e84554e3fabbd0f: Deleting consensus metadata
I20260430 19:58:59.353749 3957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "caa5357d0506482ea383b464e24f2a7a" is_pre_election: true
I20260430 19:58:59.365191 5364 leader_election.cc:290] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers caa5357d0506482ea383b464e24f2a7a (127.3.190.61:35559), dbc6c3c322c24063ad0cf6b243cf4755 (127.3.190.60:42577)
I20260430 19:58:59.367616 4039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b0a440dabed4814aaefe431944a7b5e" candidate_term: 5 candidate_status { last_received { term: 4 index: 226 } } ignore_live_leader: false dest_uuid: "dbc6c3c322c24063ad0cf6b243cf4755" is_pre_election: true
I20260430 19:58:59.369109 4953 leader_election.cc:304] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b0a440dabed4814aaefe431944a7b5e; no voters: caa5357d0506482ea383b464e24f2a7a, dbc6c3c322c24063ad0cf6b243cf4755
I20260430 19:58:59.371325 5364 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 6b0a440dabed4814aaefe431944a7b5e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:58:59.373715 4729 tablet_bootstrap.cc:492] T bdc476c6909d4c998f8b102029eb62a4 P 8a11a9a7f244435b8e84554e3fabbd0f: Bootstrap starting.
W20260430 19:58:59.381120 4954 connection.cc:570] server connection from 127.3.190.2:49631 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:59.382006 3934 connection.cc:570] server connection from 127.3.190.2:47475 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:59.382689 4470 connection.cc:570] client connection to 127.3.190.2:43939 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260430 19:58:59.382762 4729 tablet_bootstrap.cc:654] T bdc476c6909d4c998f8b102029eb62a4 P 8a11a9a7f244435b8e84554e3fabbd0f: Neither blocks nor log segments found. Creating new log.
W20260430 19:58:59.398093 5350 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:58:59.394010 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.2:43939 (local address 127.3.190.3:42577)
0430 19:58:59.394682 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0430 19:58:59.394748 (+ 66us) client_negotiation.cc:175] Beginning negotiation
0430 19:58:59.395200 (+ 452us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0430 19:58:59.397146 (+ 1946us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.2:43939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":70}
I20260430 19:58:59.387192 5313 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53008 (request call id 251) took 1059 ms. Trace:
I20260430 19:58:59.399556 5313 rpcz_store.cc:276] 0430 19:58:58.327547 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:58:58.328617 (+ 1070us) service_pool.cc:225] Handling call
0430 19:58:59.115569 (+786952us) ts_tablet_manager.cc:1151] Deleting tablet b6acec54bc7b4692894333e0150a1aff
0430 19:58:59.115612 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0430 19:58:59.251332 (+135720us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:59.323219 (+ 71887us) tablet_metadata.cc:665] Metadata flushed
0430 19:58:59.387071 (+ 63852us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":786389}
I20260430 19:58:59.401561 4013 catalog_manager.cc:5002] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953): tablet b6acec54bc7b4692894333e0150a1aff (table default.Table_16cd323d5c83450f8e84d0eb40d80d10 [id=9d892cfdee0b4827bed9dee0ae6761cd]) successfully deleted
I20260430 19:58:59.401521 5313 tablet_replica.cc:333] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: stopping tablet replica
I20260430 19:58:59.403327 5313 raft_consensus.cc:2243] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:58:59.404237 5313 pending_rounds.cc:70] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: Trying to abort 1 pending ops.
I20260430 19:58:59.404469 5313 pending_rounds.cc:77] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7280964151451664384 op_type: NO_OP noop_request { }
I20260430 19:58:59.404958 5313 raft_consensus.cc:2889] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260430 19:58:59.405319 5313 raft_consensus.cc:2272] T 28ba03500c0540cda44debe362278e47 P 8a11a9a7f244435b8e84554e3fabbd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:58:59.405639 4729 tablet_bootstrap.cc:492] T bdc476c6909d4c998f8b102029eb62a4 P 8a11a9a7f244435b8e84554e3fabbd0f: No bootstrap required, opened a new log
I20260430 19:58:59.406847 4729 ts_tablet_manager.cc:1403] T bdc476c6909d4c998f8b102029eb62a4 P 8a11a9a7f244435b8e84554e3fabbd0f: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.007s
W20260430 19:58:59.406910 4470 consensus_peers.cc:597] T 798bb6007af849e99bf8e95d84624044 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Couldn't send request to peer d8345ec7f57c4399a8a4afe4dd43439e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.2:43939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:59.407384 4470 consensus_peers.cc:597] T 67e5cb3aaa4044eb9dd06227882e2b42 P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Couldn't send request to peer d8345ec7f57c4399a8a4afe4dd43439e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.2:43939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:59.407820 4470 consensus_peers.cc:597] T 654df09f602b4f96bcf5d13a514df55c P 8a11a9a7f244435b8e84554e3fabbd0f -> Peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Couldn't send request to peer d8345ec7f57c4399a8a4afe4dd43439e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.2:43939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260430 19:58:59.408226 4470 leader_election.cc:336] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer d8345ec7f57c4399a8a4afe4dd43439e (127.3.190.2:43939): Network error: Client connection negotiation failed: client connection to 127.3.190.2:43939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260430 19:58:59.408579 4470 leader_election.cc:304] T f5c54c37bdae458d88594a3f5c401a48 P 8a11a9a7f244435b8e84554e3fabbd0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a11a9a7f244435b8e84554e3fabbd0f; no voters: 5a7356e99e744525861e12c108cfa3fe, d8345ec7f57c4399a8a4afe4dd43439e
I20260430 19:58:59.409600 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 4444
W20260430 19:58:59.436671 4016 connection.cc:570] client connection to 127.3.190.2:43939 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:58:59.447142 4016 connection.cc:570] server connection from 127.3.190.2:49905 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260430 19:58:59.485884 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 4635
W20260430 19:58:59.486048 5355 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:58:59.484582 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.3:37953 (local address 127.0.0.1:36212)
0430 19:58:59.485181 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0430 19:58:59.485353 (+ 172us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.3:37953: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":76}
I20260430 19:58:59.498809 4028 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 66e94750de27416e9adcf60fccf2ace9 (table default.Table_63f9f99010054a52adde13533fc41269 [id=245a6d903af145969e0ff7d40eb5f1e7]). Expected version 2 got 0
I20260430 19:58:59.499375 4028 catalog_manager.cc:5763] TS 8a11a9a7f244435b8e84554e3fabbd0f (127.3.190.3:37953) does not have the latest schema for tablet 3452fa52c20142f7bf396bc182a829a7 (table default.Table_f4523dfb8ab9414bad907abc56460874 [id=69dac89c4d4a4081aed4ac000a4a51c5]). Expected version 1 got 0
W20260430 19:58:59.514809 4016 connection.cc:463] server connection from 127.3.190.3:35079 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.3.190.3:35079 (request call id 60) could send its response
I20260430 19:58:59.543488 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 3911
W20260430 19:58:59.544504 4015 consensus_peers.cc:597] T 00000000000000000000000000000000 P dbc6c3c322c24063ad0cf6b243cf4755 -> Peer 6b0a440dabed4814aaefe431944a7b5e (127.3.190.62:33781): Couldn't send request to peer 6b0a440dabed4814aaefe431944a7b5e. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.62:33781: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260430 19:58:59.585855 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 3986
2026-04-30T19:58:59Z chronyd exiting
I20260430 19:58:59.641033 3832 master-stress-test.cc:585] Tables created: 146
I20260430 19:58:59.641259 3832 master-stress-test.cc:586] Tables altered: 69
I20260430 19:58:59.641407 3832 master-stress-test.cc:587] Tables deleted: 53
I20260430 19:58:59.641544 3832 master-stress-test.cc:588] Tablets replaced: 23
I20260430 19:58:59.641687 3832 master-stress-test.cc:589] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (33469 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
2026-04-30T19:58:59Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-04-30T19:58:59Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task07fIBA/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task07fIBA/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/security.properties
2026-04-30 19:59:07: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task07fIBA/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task07fIBA/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260430 19:59:16.838665 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.62:35659
--webserver_interface=127.3.190.62
--webserver_port=0
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--hive_metastore_uris=thrift://127.0.0.1:32957
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260430 19:59:17.132058 5651 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:17.132666 5651 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:17.133036 5651 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:59:17.165747 5651 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260430 19:59:17.166003 5651 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260430 19:59:17.166203 5651 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:59:17.166397 5651 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:17.166620 5651 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:59:17.166811 5651 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:59:17.204135 5651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:32957
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.62:35659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.190.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:17.205493 5651 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:17.207180 5651 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:17.218328 5657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:17.223582 5660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:17.219694 5658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:18.330165 5659 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260430 19:59:18.330288 5651 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:59:18.335033 5651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:18.337947 5651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:18.339404 5651 hybrid_clock.cc:648] HybridClock initialized: now 1777579158339360 us; error 56 us; skew 500 ppm
I20260430 19:59:18.340225 5651 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:59:18.346621 5651 webserver.cc:492] Webserver started at http://127.3.190.62:37791/ using document root <none> and password file <none>
I20260430 19:59:18.347540 5651 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:18.347751 5651 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:18.348165 5651 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:59:18.352401 5651 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data/instance:
uuid: "f46e27ec79e44f329de14da21f60e60f"
format_stamp: "Formatted at 2026-04-30 19:59:18 on dist-test-slave-6zcr"
I20260430 19:59:18.353523 5651 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/wal/instance:
uuid: "f46e27ec79e44f329de14da21f60e60f"
format_stamp: "Formatted at 2026-04-30 19:59:18 on dist-test-slave-6zcr"
I20260430 19:59:18.360786 5651 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260430 19:59:18.366465 5667 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:18.367720 5651 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260430 19:59:18.368043 5651 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/wal
uuid: "f46e27ec79e44f329de14da21f60e60f"
format_stamp: "Formatted at 2026-04-30 19:59:18 on dist-test-slave-6zcr"
I20260430 19:59:18.368355 5651 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:18.421151 5651 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:18.422626 5651 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:18.423069 5651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:18.493008 5651 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.62:35659
I20260430 19:59:18.493086 5718 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.62:35659 every 8 connection(s)
I20260430 19:59:18.495667 5651 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/data/info.pb
I20260430 19:59:18.501287 5719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:18.506137 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 5651
I20260430 19:59:18.506680 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-0/wal/instance
I20260430 19:59:18.510205 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.61:35031
--webserver_interface=127.3.190.61
--webserver_port=0
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--hive_metastore_uris=thrift://127.0.0.1:32957
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260430 19:59:18.514860 5719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:59:18.540318 5719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } has no permanent_uuid. Determining permanent_uuid...
W20260430 19:59:18.542693 5668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
W20260430 19:59:18.546769 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35031: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:18.596297 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } attempt: 1
W20260430 19:59:18.600469 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35031: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:18.668970 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } attempt: 2
W20260430 19:59:18.673442 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35031: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:18.764884 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } attempt: 3
W20260430 19:59:18.768967 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35031: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
W20260430 19:59:18.821574 5720 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:18.822156 5720 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:18.822608 5720 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:59:18.855695 5720 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260430 19:59:18.856007 5720 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260430 19:59:18.856256 5720 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:59:18.856494 5720 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:18.856779 5720 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:59:18.857033 5720 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:59:18.894953 5720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:32957
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.61:35031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.190.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:18.896323 5720 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:18.897912 5720 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:18.908794 5729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:18.912346 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } attempt: 4
W20260430 19:59:18.917785 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35031: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:19.217239 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } attempt: 5
W20260430 19:59:19.225090 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35031: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:19.772607 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } attempt: 6
W20260430 19:59:19.799727 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.61:35031: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
W20260430 19:59:18.909368 5730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:20.050835 5732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:20.057253 5731 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1145 milliseconds
W20260430 19:59:20.057413 5720 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.149s user 0.394s sys 0.747s
W20260430 19:59:20.057724 5720 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.149s user 0.394s sys 0.747s
I20260430 19:59:20.058014 5720 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:59:20.059401 5720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:20.062660 5720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:20.064303 5720 hybrid_clock.cc:648] HybridClock initialized: now 1777579160064291 us; error 90 us; skew 500 ppm
I20260430 19:59:20.065168 5720 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:59:20.073065 5720 webserver.cc:492] Webserver started at http://127.3.190.61:45235/ using document root <none> and password file <none>
I20260430 19:59:20.074013 5720 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:20.074240 5720 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:20.074746 5720 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:59:20.079203 5720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/instance:
uuid: "f2878444c3a64840b53f8d5f6fc6211d"
format_stamp: "Formatted at 2026-04-30 19:59:20 on dist-test-slave-6zcr"
I20260430 19:59:20.080294 5720 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal/instance:
uuid: "f2878444c3a64840b53f8d5f6fc6211d"
format_stamp: "Formatted at 2026-04-30 19:59:20 on dist-test-slave-6zcr"
I20260430 19:59:20.089211 5720 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260430 19:59:20.095935 5741 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:20.097357 5720 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260430 19:59:20.097733 5720 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
uuid: "f2878444c3a64840b53f8d5f6fc6211d"
format_stamp: "Formatted at 2026-04-30 19:59:20 on dist-test-slave-6zcr"
I20260430 19:59:20.098086 5720 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:20.168721 5720 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:20.170217 5720 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:20.170735 5720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:20.240409 5720 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.61:35031
I20260430 19:59:20.240474 5792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.61:35031 every 8 connection(s)
I20260430 19:59:20.243076 5720 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
I20260430 19:59:20.243932 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 5720
I20260430 19:59:20.244340 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal/instance
I20260430 19:59:20.246819 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.60:45507
--webserver_interface=127.3.190.60
--webserver_port=0
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--hive_metastore_uris=thrift://127.0.0.1:32957
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260430 19:59:20.249045 5793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:20.260041 5793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:59:20.281483 5793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:59:20.296428 5793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } has no permanent_uuid. Determining permanent_uuid...
W20260430 19:59:20.299073 5744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 19:59:20.303402 5793 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:20.352958 5793 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 1
W20260430 19:59:20.357299 5793 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:20.425710 5793 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 2
W20260430 19:59:20.429819 5793 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:20.521238 5793 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 3
W20260430 19:59:20.526039 5793 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 19:59:20.563349 5794 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:20.563907 5794 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:20.564342 5794 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:59:20.596706 5794 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260430 19:59:20.596956 5794 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260430 19:59:20.597213 5794 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:59:20.597448 5794 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:20.597679 5794 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:59:20.597910 5794 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:59:20.634811 5794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:32957
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.60:45507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.190.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:20.636103 5794 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:20.637668 5794 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:20.648653 5804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:20.669536 5793 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 4
W20260430 19:59:20.675696 5793 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:20.860284 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } attempt: 7
I20260430 19:59:20.897547 5719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } has no permanent_uuid. Determining permanent_uuid...
W20260430 19:59:20.908210 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:20.966699 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 1
W20260430 19:59:20.972704 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:20.975239 5793 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 5
W20260430 19:59:20.981631 5793 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:21.054257 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 2
W20260430 19:59:21.060242 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:21.145721 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 3
W20260430 19:59:21.151443 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:21.291932 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 4
W20260430 19:59:21.298393 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:21.529117 5793 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 6
W20260430 19:59:21.564644 5793 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:21.581979 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 5
W20260430 19:59:21.588728 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 19:59:22.051780 5803 debug-util.cc:398] Leaking SignalData structure 0x7b080002ecc0 after lost signal to thread 5794
W20260430 19:59:22.134006 5794 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.486s user 0.598s sys 0.882s
W20260430 19:59:22.134388 5794 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.486s user 0.598s sys 0.882s
W20260430 19:59:20.649257 5805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:22.136668 5807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:22.139284 5806 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1485 milliseconds
I20260430 19:59:22.139331 5794 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:59:22.140539 5794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:22.141214 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 6
I20260430 19:59:22.143335 5794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:22.144850 5794 hybrid_clock.cc:648] HybridClock initialized: now 1777579162144743 us; error 103 us; skew 500 ppm
I20260430 19:59:22.145967 5794 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260430 19:59:22.148110 5719 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.190.60:45507: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:22.152482 5794 webserver.cc:492] Webserver started at http://127.3.190.60:37189/ using document root <none> and password file <none>
I20260430 19:59:22.153421 5794 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:22.153662 5794 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:22.154141 5794 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:59:22.160451 5794 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/instance:
uuid: "867d1f5512324df489edc5a61b79df54"
format_stamp: "Formatted at 2026-04-30 19:59:22 on dist-test-slave-6zcr"
I20260430 19:59:22.161756 5794 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal/instance:
uuid: "867d1f5512324df489edc5a61b79df54"
format_stamp: "Formatted at 2026-04-30 19:59:22 on dist-test-slave-6zcr"
I20260430 19:59:22.169960 5794 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.009s sys 0.001s
I20260430 19:59:22.175940 5822 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:22.177186 5794 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260430 19:59:22.177525 5794 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
uuid: "867d1f5512324df489edc5a61b79df54"
format_stamp: "Formatted at 2026-04-30 19:59:22 on dist-test-slave-6zcr"
I20260430 19:59:22.177877 5794 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:22.224786 5794 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:22.226266 5794 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:22.226729 5794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:22.296981 5794 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.60:45507
I20260430 19:59:22.297098 5873 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.60:45507 every 8 connection(s)
I20260430 19:59:22.299949 5794 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
I20260430 19:59:22.305356 5874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:22.310557 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 5794
I20260430 19:59:22.311002 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal/instance
I20260430 19:59:22.317176 5874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:59:22.349014 5874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:59:22.358439 5874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } has no permanent_uuid. Determining permanent_uuid...
I20260430 19:59:22.374258 5874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Bootstrap starting.
I20260430 19:59:22.380501 5874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:22.382309 5874 log.cc:826] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Log is configured to *not* fsync() on all Append() calls
I20260430 19:59:22.386533 5874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: No bootstrap required, opened a new log
I20260430 19:59:22.406196 5874 raft_consensus.cc:359] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:22.406873 5874 raft_consensus.cc:385] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:22.407105 5874 raft_consensus.cc:740] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867d1f5512324df489edc5a61b79df54, State: Initialized, Role: FOLLOWER
I20260430 19:59:22.407786 5874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:22.410116 5882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:22.410733 5882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:22.411805 5874 sys_catalog.cc:565] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: configured and running, proceeding with master startup.
W20260430 19:59:22.442655 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.443104 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.457621 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.458388 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.437431 5794 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: OK, instance_id { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" instance_seqno: 1777579158474141 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260430 19:59:22.465849 5794 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 19:59:22.473879 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.476047 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.481665 5828 tablet.cc:2404] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260430 19:59:22.490654 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.491153 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.505575 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.505991 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.520496 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.521042 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.535615 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.535964 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.550262 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.550664 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.566717 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.567189 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.581583 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.581947 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.596318 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.596664 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.611068 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.611485 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.625157 5793 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 7
W20260430 19:59:22.625802 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.626101 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.642356 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.642748 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.643371 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Bootstrap starting.
I20260430 19:59:22.648993 5793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:22.650765 5793 log.cc:826] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Log is configured to *not* fsync() on all Append() calls
I20260430 19:59:22.655125 5793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: No bootstrap required, opened a new log
W20260430 19:59:22.657110 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.657413 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.671713 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.672024 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.674234 5793 raft_consensus.cc:359] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:22.674857 5793 raft_consensus.cc:385] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:22.675088 5793 raft_consensus.cc:740] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2878444c3a64840b53f8d5f6fc6211d, State: Initialized, Role: FOLLOWER
I20260430 19:59:22.675734 5793 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:22.678385 5902 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:22.679215 5902 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:22.679998 5793 sys_catalog.cc:565] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: configured and running, proceeding with master startup.
W20260430 19:59:22.686259 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.686570 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.701087 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.701469 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.720644 5882 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:22.721169 5882 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:22.723510 5882 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
W20260430 19:59:22.727502 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.727943 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.729240 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:22.730135 5768 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867d1f5512324df489edc5a61b79df54 in term 0.
I20260430 19:59:22.734544 5823 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters:
W20260430 19:59:22.734990 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260430 19:59:22.735601 5882 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:22.735961 5882 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:22.736271 5882 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:59:22.746979 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.747303 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.735606 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.755834 5882 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:22.757814 5882 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 1 election: Requested vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
W20260430 19:59:22.762467 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.763073 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.767580 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260430 19:59:22.769411 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d"
I20260430 19:59:22.769975 5768 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:59:22.775475 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.778007 5768 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867d1f5512324df489edc5a61b79df54 in term 1.
I20260430 19:59:22.779181 5823 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters:
I20260430 19:59:22.780109 5882 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:22.720299 5720 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: OK, instance_id { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" instance_seqno: 1777579158474141 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260430 19:59:22.793282 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260430 19:59:22.794736 5882 raft_consensus.cc:697] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 1 LEADER]: Becoming Leader. State: Replica: 867d1f5512324df489edc5a61b79df54, State: Running, Role: LEADER
I20260430 19:59:22.794996 5720 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 19:59:22.795066 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.795846 5882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
W20260430 19:59:22.815227 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.816108 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.833998 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.834488 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.826378 5919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 1 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:22.837527 5919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
W20260430 19:59:22.849449 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.849766 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.857797 5896 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.858235 5896 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.859793 5921 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:59:22.870639 5921 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20260430 19:59:22.871637 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.871954 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.886412 5917 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260430 19:59:22.887225 5917 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:22.887662 5768 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Refusing update from remote peer 867d1f5512324df489edc5a61b79df54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:59:22.889513 5882 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:59:22.928808 5747 tablet.cc:2404] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:59:22.931491 5902 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 1 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:22.932406 5902 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:22.939448 5922 mvcc.cc:204] Tried to move back new op lower bound from 7280964251168481280 to 7280964250823135232. Current Snapshot: MvccSnapshot[applied={T|T < 7280964251168481280}]
I20260430 19:59:22.946413 5919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:22.947191 5919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:22.949054 5902 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:22.949863 5902 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:22.952703 5917 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:22.952952 5917 catalog_manager.cc:1562] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: success
I20260430 19:59:22.958841 5921 catalog_manager.cc:1357] Generated new cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:22.959159 5921 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
W20260430 19:59:22.959951 5917 catalog_manager.cc:1586] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260430 19:59:22.960239 5917 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260430 19:59:22.984761 5917 catalog_manager.cc:1586] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260430 19:59:22.985657 5917 catalog_manager.cc:883] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260430 19:59:23.000976 5921 catalog_manager.cc:1380] Generated new certificate authority record
I20260430 19:59:23.006803 5921 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:59:23.010027 5917 catalog_manager.cc:1584] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: acquiring CA information for follower catalog manager: success
W20260430 19:59:23.014809 5917 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
W20260430 19:59:23.045759 5917 catalog_manager.cc:883] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260430 19:59:23.050045 5921 catalog_manager.cc:6044] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Generated new TSK 0
I20260430 19:59:23.052023 5921 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:59:23.058809 5921 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260430 19:59:23.063258 5921 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260430 19:59:23.067703 5917 catalog_manager.cc:1612] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:59:23.101919 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.190.1:0
--local_ip_for_outbound_sockets=127.3.190.1
--webserver_interface=127.3.190.1
--webserver_port=0
--tserver_master_addrs=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260430 19:59:23.181468 5719 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } attempt: 7
I20260430 19:59:23.245477 5719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Bootstrap starting.
I20260430 19:59:23.269583 5719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:23.277262 5719 log.cc:826] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Log is configured to *not* fsync() on all Append() calls
W20260430 19:59:23.279608 5823 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.190.62:35659 timed out after 0.520s (SENT)
I20260430 19:59:23.322413 5719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: No bootstrap required, opened a new log
I20260430 19:59:23.435236 5719 raft_consensus.cc:359] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:23.440337 5719 raft_consensus.cc:385] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:23.441646 5719 raft_consensus.cc:740] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46e27ec79e44f329de14da21f60e60f, State: Initialized, Role: FOLLOWER
I20260430 19:59:23.445741 5719 consensus_queue.cc:260] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:23.519634 5933 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:23.520462 5933 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:23.529950 5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:23.531092 5692 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:59:23.531252 5694 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49298 (request call id 2) took 803 ms (client timeout 650 ms). Trace:
W20260430 19:59:23.531591 5694 rpcz_store.cc:269] 0430 19:59:22.727624 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:59:22.727763 (+ 139us) service_pool.cc:225] Handling call
0430 19:59:23.531173 (+803410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260430 19:59:23.533841 5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f"
I20260430 19:59:23.529942 5690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:23.537211 5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f"
W20260430 19:59:23.539680 5693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49298 (request call id 3) took 776 ms (client timeout 520 ms). Trace:
W20260430 19:59:23.540014 5693 rpcz_store.cc:269] 0430 19:59:22.762971 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:59:22.763139 (+ 168us) service_pool.cc:225] Handling call
0430 19:59:23.539621 (+776482us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260430 19:59:23.549297 5692 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Refusing update from remote peer 867d1f5512324df489edc5a61b79df54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260430 19:59:23.550252 5933 raft_consensus.cc:493] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:23.550779 5719 sys_catalog.cc:565] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: configured and running, proceeding with master startup.
I20260430 19:59:23.551211 5933 raft_consensus.cc:515] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:23.560410 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:23.561276 5768 raft_consensus.cc:2410] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46e27ec79e44f329de14da21f60e60f for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260430 19:59:23.562214 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:23.558545 5933 leader_election.cc:290] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:23.566783 5668 leader_election.cc:304] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f46e27ec79e44f329de14da21f60e60f; no voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d
I20260430 19:59:23.594802 5933 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260430 19:59:23.596570 5935 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260430 19:59:23.598532 5823 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.62:35659, user_credentials={real_user=slave}} blocked reactor thread for 45678.5us
I20260430 19:59:23.611261 5934 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:23.611817 5934 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:23.615487 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:23.615541 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:23.616145 5692 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2878444c3a64840b53f8d5f6fc6211d in term 1.
I20260430 19:59:23.617697 5742 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d, f46e27ec79e44f329de14da21f60e60f; no voters: 867d1f5512324df489edc5a61b79df54
I20260430 19:59:23.634691 5934 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:23.635404 5934 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260430 19:59:23.635800 5934 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:23.636377 5934 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 1 FOLLOWER]: Advancing to term 2
W20260430 19:59:23.663829 5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.190.62:35659, user_credentials={real_user=slave}} blocked reactor thread for 46352.5us
I20260430 19:59:23.658209 5934 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:23.692634 5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f"
I20260430 19:59:23.692973 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54"
I20260430 19:59:23.694468 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:23.695526 5941 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260430 19:59:23.700280 5934 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 2 election: Requested vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:23.701825 5933 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 1 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:23.702008 5946 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:23.702510 5933 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:23.702755 5946 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:23.758494 5958 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260430 19:59:23.778295 5651 master_runner.cc:232] Existing masters have differing terms: 2 vs 1
I20260430 19:59:23.778842 5651 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:59:23.794363 5958 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:23.794684 5958 catalog_manager.cc:1562] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: loading cluster ID for follower catalog manager: success
I20260430 19:59:23.817858 5958 catalog_manager.cc:1584] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: acquiring CA information for follower catalog manager: success
I20260430 19:59:23.837965 5958 catalog_manager.cc:1612] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:59:24.020905 5935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:24.021633 5935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
W20260430 19:59:24.083175 5928 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:24.083909 5928 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:24.084872 5928 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260430 19:59:24.119562 5768 raft_consensus.cc:1240] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Rejecting Update request from peer 867d1f5512324df489edc5a61b79df54 for earlier term 1. Current term is 2. Ops: []
I20260430 19:59:24.121206 5935 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260430 19:59:24.122167 5935 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 1 LEADER]: Stepping down as leader of term 1
I20260430 19:59:24.122459 5935 raft_consensus.cc:740] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 867d1f5512324df489edc5a61b79df54, State: Running, Role: LEADER
I20260430 19:59:24.123430 5935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:24.124936 5935 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 1 FOLLOWER]: Advancing to term 2
I20260430 19:59:24.138001 5896 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260430 19:59:24.160597 5928 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:24.162319 5928 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.190.1
I20260430 19:59:24.234300 5928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.190.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.190.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.190.1
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:24.236304 5928 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:24.255800 5928 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:24.304104 5966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:24.450235 5972 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:24.453022 5972 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:24.463400 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:24.465627 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:24.466296 5849 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2878444c3a64840b53f8d5f6fc6211d in term 2.
I20260430 19:59:24.470157 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters: f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:24.475867 5972 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:24.476940 5972 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260430 19:59:24.479555 5972 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:24.480566 5972 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 2 FOLLOWER]: Advancing to term 3
I20260430 19:59:24.493739 5972 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:24.497767 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f"
I20260430 19:59:24.499768 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54"
I20260430 19:59:24.500332 5849 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 2 FOLLOWER]: Advancing to term 3
I20260430 19:59:24.509550 5849 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2878444c3a64840b53f8d5f6fc6211d in term 3.
I20260430 19:59:24.512041 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters: f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:24.513145 5973 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 3 FOLLOWER]: Leader election won for term 3
I20260430 19:59:24.515900 5972 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 3 election: Requested vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:24.528173 5975 raft_consensus.cc:493] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:24.528695 5973 raft_consensus.cc:697] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 3 LEADER]: Becoming Leader. State: Replica: f2878444c3a64840b53f8d5f6fc6211d, State: Running, Role: LEADER
I20260430 19:59:24.529699 5973 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:24.533455 5975 raft_consensus.cc:515] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:24.538013 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:24.541538 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:24.542250 5849 raft_consensus.cc:2368] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46e27ec79e44f329de14da21f60e60f for earlier term 2. Current term is 3.
I20260430 19:59:24.544486 5670 leader_election.cc:400] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46e27ec79e44f329de14da21f60e60f for earlier term 2. Current term is 3.
I20260430 19:59:24.545135 5670 leader_election.cc:403] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260430 19:59:24.551147 5975 leader_election.cc:290] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:24.553180 5975 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 1 FOLLOWER]: Advancing to term 3
I20260430 19:59:24.568250 5972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2878444c3a64840b53f8d5f6fc6211d. Latest consensus state: current_term: 3 leader_uuid: "f2878444c3a64840b53f8d5f6fc6211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:24.568962 5972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:24.585139 5978 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:59:24.587272 5975 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f46e27ec79e44f329de14da21f60e60f for earlier term 2. Current term is 3.
I20260430 19:59:24.600756 5978 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:59:24.606612 5978 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:24.606993 5978 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:59:24.614796 5978 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:59:24.620903 5978 catalog_manager.cc:6055] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Loaded TSK: 0
I20260430 19:59:24.622572 5978 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:59:24.627933 5978 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260430 19:59:24.633354 5978 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260430 19:59:25.067281 5979 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:25.067900 5979 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:25.069802 5979 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
I20260430 19:59:25.071308 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:25.071902 5692 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867d1f5512324df489edc5a61b79df54 in term 3.
I20260430 19:59:25.072862 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:25.075439 5823 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f; no voters: f2878444c3a64840b53f8d5f6fc6211d
I20260430 19:59:25.076337 5979 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260430 19:59:25.076706 5979 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:25.077080 5979 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 3 FOLLOWER]: Advancing to term 4
I20260430 19:59:25.092093 5979 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:25.094039 5979 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 4 election: Requested vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
I20260430 19:59:25.096050 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f"
I20260430 19:59:25.096568 5692 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 3 FOLLOWER]: Advancing to term 4
I20260430 19:59:25.107522 5692 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867d1f5512324df489edc5a61b79df54 in term 4.
I20260430 19:59:25.110960 5823 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f; no voters:
I20260430 19:59:25.112537 5979 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 FOLLOWER]: Leader election won for term 4
I20260430 19:59:25.126871 5692 raft_consensus.cc:1240] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Rejecting Update request from peer f2878444c3a64840b53f8d5f6fc6211d for earlier term 3. Current term is 4. Ops: []
I20260430 19:59:25.132143 5979 raft_consensus.cc:697] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 LEADER]: Becoming Leader. State: Replica: 867d1f5512324df489edc5a61b79df54, State: Running, Role: LEADER
I20260430 19:59:25.133189 5979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:25.136186 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d"
I20260430 19:59:25.140887 5982 consensus_queue.cc:1059] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260430 19:59:25.149884 5980 raft_consensus.cc:3055] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 3 LEADER]: Stepping down as leader of term 3
I20260430 19:59:25.149935 5981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 4 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:25.150323 5980 raft_consensus.cc:740] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f2878444c3a64840b53f8d5f6fc6211d, State: Running, Role: LEADER
I20260430 19:59:25.150671 5981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:25.151225 5980 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:25.159817 5980 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 3 FOLLOWER]: Advancing to term 4
I20260430 19:59:25.161173 5985 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:59:25.186414 5985 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:59:25.200588 5985 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:25.200987 5985 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:59:25.214188 5985 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:59:25.223562 5985 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Loaded TSK: 0
I20260430 19:59:25.229993 5985 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:59:25.240181 5985 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260430 19:59:25.248816 5985 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260430 19:59:25.592096 5692 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Refusing update from remote peer 867d1f5512324df489edc5a61b79df54: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260430 19:59:25.593575 5981 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260430 19:59:25.626421 5986 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 4 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:25.633292 5986 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:25.647418 5979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:25.647835 5768 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Refusing update from remote peer 867d1f5512324df489edc5a61b79df54: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260430 19:59:25.648103 5979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:25.648166 5768 pending_rounds.cc:85] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Aborting all ops after (but not including) 4
I20260430 19:59:25.648447 5768 pending_rounds.cc:107] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260430 19:59:25.648702 5768 raft_consensus.cc:2889] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260430 19:59:25.650947 5981 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260430 19:59:25.653183 5986 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:25.657582 5986 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:25.676141 5980 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:25.676807 5980 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:25.677708 5984 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 4 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:25.678356 5984 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
W20260430 19:59:24.335599 5967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:25.826864 5968 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection timed out after 1507 milliseconds
I20260430 19:59:25.827448 5928 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260430 19:59:24.325747 5970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:25.833685 5928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:25.836387 5928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:25.837929 5928 hybrid_clock.cc:648] HybridClock initialized: now 1777579165837899 us; error 55 us; skew 500 ppm
I20260430 19:59:25.838930 5928 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:59:25.846310 5928 webserver.cc:492] Webserver started at http://127.3.190.1:39775/ using document root <none> and password file <none>
I20260430 19:59:25.847296 5928 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:25.847496 5928 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:25.847970 5928 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:59:25.853045 5928 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data/instance:
uuid: "499a4336efe645d4926c94600c732b82"
format_stamp: "Formatted at 2026-04-30 19:59:25 on dist-test-slave-6zcr"
I20260430 19:59:25.854188 5928 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/instance:
uuid: "499a4336efe645d4926c94600c732b82"
format_stamp: "Formatted at 2026-04-30 19:59:25 on dist-test-slave-6zcr"
I20260430 19:59:25.862344 5928 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260430 19:59:25.870260 5996 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:25.871698 5928 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260430 19:59:25.872218 5928 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal
uuid: "499a4336efe645d4926c94600c732b82"
format_stamp: "Formatted at 2026-04-30 19:59:25 on dist-test-slave-6zcr"
I20260430 19:59:25.872649 5928 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:25.934515 5928 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:25.936616 5928 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:25.937208 5928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:25.941627 5928 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260430 19:59:25.947763 5928 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260430 19:59:25.948062 5928 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:25.948367 5928 ts_tablet_manager.cc:616] Registered 0 tablets
I20260430 19:59:25.948587 5928 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:26.147498 5928 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.1:33861
I20260430 19:59:26.147647 6108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.1:33861 every 8 connection(s)
I20260430 19:59:26.150705 5928 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/data/info.pb
I20260430 19:59:26.156661 5984 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:26.157032 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 5928
I20260430 19:59:26.157513 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/instance
I20260430 19:59:26.157187 5984 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:26.159148 5984 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:26.161267 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:26.162243 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:26.169873 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:26.175952 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.190.2:0
--local_ip_for_outbound_sockets=127.3.190.2
--webserver_interface=127.3.190.2
--webserver_port=0
--tserver_master_addrs=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260430 19:59:26.177362 5980 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:59:26.237016 6109 heartbeater.cc:344] Connected to a master server at 127.3.190.60:45507
I20260430 19:59:26.237547 6109 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:26.238862 6109 heartbeater.cc:507] Master 127.3.190.60:45507 requested a full tablet report, sending...
I20260430 19:59:26.241962 5839 ts_manager.cc:194] Registered new tserver with Master: 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861)
I20260430 19:59:26.245731 5839 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.190.1:42609
I20260430 19:59:26.253433 6111 heartbeater.cc:344] Connected to a master server at 127.3.190.62:35659
I20260430 19:59:26.253839 6111 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:26.254522 6111 heartbeater.cc:507] Master 127.3.190.62:35659 requested a full tablet report, sending...
I20260430 19:59:26.257853 5684 ts_manager.cc:194] Registered new tserver with Master: 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861)
I20260430 19:59:26.264029 6110 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35031
I20260430 19:59:26.264384 6110 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:26.265074 6110 heartbeater.cc:507] Master 127.3.190.61:35031 requested a full tablet report, sending...
I20260430 19:59:26.264798 5981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:26.265551 5981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:26.268020 5758 ts_manager.cc:194] Registered new tserver with Master: 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861)
W20260430 19:59:26.589048 6116 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:26.589543 6116 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:26.590147 6116 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:59:26.628556 6116 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:26.629468 6116 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.190.2
I20260430 19:59:26.669755 6116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.190.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.190.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.190.2
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:26.671303 6116 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:26.672909 6116 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:26.687145 6130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:26.750046 6109 heartbeater.cc:499] Master 127.3.190.60:45507 was elected leader, sending a full tablet report...
I20260430 19:59:27.727061 6137 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:27.727897 6137 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:27.731464 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:27.735242 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:27.737076 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:27.735246 6137 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:27.740621 6137 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:59:28.051218 6139 raft_consensus.cc:493] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:28.051808 6139 raft_consensus.cc:515] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:28.054922 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:28.058707 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:28.061621 5670 leader_election.cc:304] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f46e27ec79e44f329de14da21f60e60f; no voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d
I20260430 19:59:28.068092 6139 leader_election.cc:290] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:28.068693 6139 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260430 19:59:28.244939 6116 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.556s user 0.531s sys 0.978s
W20260430 19:59:28.245747 6116 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.557s user 0.531s sys 0.978s
W20260430 19:59:26.688197 6131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:28.089691 6129 debug-util.cc:398] Leaking SignalData structure 0x7b0800028ca0 after lost signal to thread 6116
W20260430 19:59:28.247578 6133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:28.250759 6132 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1556 milliseconds
I20260430 19:59:28.250880 6116 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:59:28.252353 6116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:28.254964 6116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:28.256412 6116 hybrid_clock.cc:648] HybridClock initialized: now 1777579168256355 us; error 61 us; skew 500 ppm
I20260430 19:59:28.257266 6116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:59:28.263623 6116 webserver.cc:492] Webserver started at http://127.3.190.2:39695/ using document root <none> and password file <none>
I20260430 19:59:28.264590 6116 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:28.264814 6116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:28.265247 6116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:59:28.270535 6116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data/instance:
uuid: "787275a4c5c64a6aa81be209814867bf"
format_stamp: "Formatted at 2026-04-30 19:59:28 on dist-test-slave-6zcr"
I20260430 19:59:28.271858 6116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/instance:
uuid: "787275a4c5c64a6aa81be209814867bf"
format_stamp: "Formatted at 2026-04-30 19:59:28 on dist-test-slave-6zcr"
I20260430 19:59:28.279830 6116 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260430 19:59:28.286197 6145 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:28.287427 6116 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260430 19:59:28.287817 6116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal
uuid: "787275a4c5c64a6aa81be209814867bf"
format_stamp: "Formatted at 2026-04-30 19:59:28 on dist-test-slave-6zcr"
I20260430 19:59:28.288326 6116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:28.289950 6146 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:28.290373 6146 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:28.291796 6146 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:28.292608 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:28.292737 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:28.293988 5742 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:28.294903 6146 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:59:28.354822 6116 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:28.356343 6116 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:28.356830 6116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:28.359427 6116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260430 19:59:28.363593 6116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260430 19:59:28.363812 6116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:28.364056 6116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260430 19:59:28.364212 6116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:28.506001 6258 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.2:35955 every 8 connection(s)
I20260430 19:59:28.506181 6116 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.2:35955
I20260430 19:59:28.508980 6116 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/data/info.pb
I20260430 19:59:28.510106 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 6116
I20260430 19:59:28.510888 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/instance
I20260430 19:59:28.527956 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.190.3:0
--local_ip_for_outbound_sockets=127.3.190.3
--webserver_interface=127.3.190.3
--webserver_port=0
--tserver_master_addrs=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260430 19:59:28.570223 6260 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35031
I20260430 19:59:28.570858 6260 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:28.572219 6260 heartbeater.cc:507] Master 127.3.190.61:35031 requested a full tablet report, sending...
I20260430 19:59:28.575794 5758 ts_manager.cc:194] Registered new tserver with Master: 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:28.577405 6259 heartbeater.cc:344] Connected to a master server at 127.3.190.60:45507
I20260430 19:59:28.577766 6259 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:28.578701 6259 heartbeater.cc:507] Master 127.3.190.60:45507 requested a full tablet report, sending...
I20260430 19:59:28.579555 6263 heartbeater.cc:344] Connected to a master server at 127.3.190.62:35659
I20260430 19:59:28.579921 6263 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:28.580830 6263 heartbeater.cc:507] Master 127.3.190.62:35659 requested a full tablet report, sending...
I20260430 19:59:28.580873 5839 ts_manager.cc:194] Registered new tserver with Master: 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:28.582507 5839 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.190.2:46363
I20260430 19:59:28.583549 5684 ts_manager.cc:194] Registered new tserver with Master: 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 19:59:28.872059 6265 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:28.872514 6265 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:28.873026 6265 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:59:28.910812 6265 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:28.912025 6265 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.190.3
I20260430 19:59:28.951711 6265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.190.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.190.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.190.3
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:28.953052 6265 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:28.955013 6265 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:28.968099 6276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:29.085690 6259 heartbeater.cc:499] Master 127.3.190.60:45507 was elected leader, sending a full tablet report...
I20260430 19:59:29.286144 6282 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:29.288597 6282 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:29.304625 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:29.309036 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:29.313390 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:29.321945 6282 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:29.325822 6282 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:59:29.888029 6284 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:29.888659 6284 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:29.891986 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:29.895772 5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:29.898447 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:29.901486 6284 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:29.902558 6284 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260430 19:59:30.368923 6275 debug-util.cc:398] Leaking SignalData structure 0x7b0800028ca0 after lost signal to thread 6265
W20260430 19:59:30.497785 6275 kernel_stack_watchdog.cc:198] Thread 6265 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 399ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260430 19:59:28.968962 6277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:30.498992 6265 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.531s user 0.609s sys 0.871s
W20260430 19:59:30.499326 6265 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.532s user 0.609s sys 0.871s
W20260430 19:59:30.499946 6279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:30.503419 6278 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1530 milliseconds
I20260430 19:59:30.503494 6265 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:59:30.504854 6265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:30.513679 6265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:30.515151 6265 hybrid_clock.cc:648] HybridClock initialized: now 1777579170515104 us; error 54 us; skew 500 ppm
I20260430 19:59:30.516599 6265 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:59:30.522922 6265 webserver.cc:492] Webserver started at http://127.3.190.3:35799/ using document root <none> and password file <none>
I20260430 19:59:30.523970 6265 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:30.524184 6265 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:30.524608 6265 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260430 19:59:30.529841 6265 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data/instance:
uuid: "cd13173a6c184acaa26b63bdbf8d6656"
format_stamp: "Formatted at 2026-04-30 19:59:30 on dist-test-slave-6zcr"
I20260430 19:59:30.531104 6265 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/instance:
uuid: "cd13173a6c184acaa26b63bdbf8d6656"
format_stamp: "Formatted at 2026-04-30 19:59:30 on dist-test-slave-6zcr"
I20260430 19:59:30.539597 6265 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260430 19:59:30.545650 6291 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:30.546824 6265 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260430 19:59:30.547119 6265 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal
uuid: "cd13173a6c184acaa26b63bdbf8d6656"
format_stamp: "Formatted at 2026-04-30 19:59:30 on dist-test-slave-6zcr"
I20260430 19:59:30.548818 6265 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:30.597847 6265 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:30.599402 6265 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:30.599833 6265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:30.602537 6265 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260430 19:59:30.606611 6265 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260430 19:59:30.606823 6265 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260430 19:59:30.607061 6265 ts_tablet_manager.cc:616] Registered 0 tablets
I20260430 19:59:30.607205 6265 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:30.749224 6265 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.3:45977
I20260430 19:59:30.749349 6403 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.3:45977 every 8 connection(s)
I20260430 19:59:30.751746 6265 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/data/info.pb
I20260430 19:59:30.756109 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 6265
I20260430 19:59:30.756656 3832 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/instance
I20260430 19:59:30.789283 6404 heartbeater.cc:344] Connected to a master server at 127.3.190.60:45507
I20260430 19:59:30.789822 6404 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:30.791330 6404 heartbeater.cc:507] Master 127.3.190.60:45507 requested a full tablet report, sending...
I20260430 19:59:30.798784 5839 ts_manager.cc:194] Registered new tserver with Master: cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:30.799950 6405 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35031
I20260430 19:59:30.800405 6405 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:30.800720 6407 heartbeater.cc:344] Connected to a master server at 127.3.190.62:35659
I20260430 19:59:30.800751 5839 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.190.3:41517
I20260430 19:59:30.801056 6407 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:30.802474 6405 heartbeater.cc:507] Master 127.3.190.61:35031 requested a full tablet report, sending...
I20260430 19:59:30.802820 6407 heartbeater.cc:507] Master 127.3.190.62:35659 requested a full tablet report, sending...
I20260430 19:59:30.805121 5758 ts_manager.cc:194] Registered new tserver with Master: cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:30.805346 5684 ts_manager.cc:194] Registered new tserver with Master: cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:30.820904 3832 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260430 19:59:30.865605 5839 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57008:
name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:30.865404 5838 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57008:
name: "default.Table_1f40269e3dc346979445899c9c93ac54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:30.868549 5837 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57008:
name: "default.Table_5ef46544dc424dea9756db410f1a18c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:30.870648 5836 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57008:
name: "default.Table_a4a99cd76d004be8a0e4b66d9e74fa77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:59:30.876235 5839 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:59:30.882560 5837 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5ef46544dc424dea9756db410f1a18c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:59:30.883307 5838 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1f40269e3dc346979445899c9c93ac54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:59:30.884297 5836 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:59:30.927021 3832 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task07fIBA/build/tsan/bin/kudu with pid 5794
I20260430 19:59:30.928491 6436 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:30.929029 6436 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:30.931473 6436 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:30.931741 5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f46e27ec79e44f329de14da21f60e60f" is_pre_election: true
I20260430 19:59:30.949362 6439 raft_consensus.cc:493] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
W20260430 19:59:30.955417 6416 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.190.60:45507)
W20260430 19:59:30.961046 5744 connection.cc:570] client connection to 127.3.190.60:45507 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:30.961499 5744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 6 similar messages]
I20260430 19:59:30.961763 6439 raft_consensus.cc:515] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:30.966776 6439 leader_election.cc:290] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:30.969156 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:30.969883 5768 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46e27ec79e44f329de14da21f60e60f in term 4.
W20260430 19:59:30.971670 6420 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:59:30.958527 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.60:45507 (local address 127.0.0.1:57018)
0430 19:59:30.959367 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0430 19:59:30.959410 (+ 43us) client_negotiation.cc:175] Beginning negotiation
0430 19:59:30.959761 (+ 351us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0430 19:59:30.970051 (+ 10290us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":193}
W20260430 19:59:30.988593 6443 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:59:30.969215 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.60:45507 (local address 127.0.0.1:57034)
0430 19:59:30.987147 (+ 17932us) negotiation.cc:107] Waiting for socket to connect
0430 19:59:30.987413 (+ 266us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":17265,"threads_started":1}
I20260430 19:59:30.990435 5668 leader_election.cc:304] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d, f46e27ec79e44f329de14da21f60e60f; no voters:
W20260430 19:59:30.989696 5744 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection reset by peer (error 104)
I20260430 19:59:30.991485 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:30.992383 6436 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:59:30.993372 6439 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260430 19:59:30.993711 6439 raft_consensus.cc:493] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:30.994097 6439 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 4 FOLLOWER]: Advancing to term 5
W20260430 19:59:30.995251 6442 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:59:30.967241 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.60:45507 (local address 127.0.0.1:57022)
0430 19:59:30.993799 (+ 26558us) negotiation.cc:107] Waiting for socket to connect
0430 19:59:30.994084 (+ 285us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":15953,"thread_start_us":16531,"threads_started":1}
W20260430 19:59:30.998394 5670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection reset by peer (error 104) [suppressed 13 similar messages]
I20260430 19:59:31.019497 6439 raft_consensus.cc:515] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:31.026489 5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f46e27ec79e44f329de14da21f60e60f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d"
I20260430 19:59:31.027060 5768 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 4 FOLLOWER]: Advancing to term 5
I20260430 19:59:31.040520 6439 leader_election.cc:290] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 election: Requested vote from peers f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:31.043473 5670 leader_election.cc:336] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:31.044400 5768 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46e27ec79e44f329de14da21f60e60f in term 5.
I20260430 19:59:31.049867 5668 leader_election.cc:304] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d, f46e27ec79e44f329de14da21f60e60f; no voters:
I20260430 19:59:31.050693 6439 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 5 FOLLOWER]: Leader election won for term 5
W20260430 19:59:31.051910 5670 leader_election.cc:336] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:31.059619 6439 raft_consensus.cc:697] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [term 5 LEADER]: Becoming Leader. State: Replica: f46e27ec79e44f329de14da21f60e60f, State: Running, Role: LEADER
I20260430 19:59:31.060771 6439 consensus_queue.cc:237] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:31.073230 6445 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f46e27ec79e44f329de14da21f60e60f. Latest consensus state: current_term: 5 leader_uuid: "f46e27ec79e44f329de14da21f60e60f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:31.074265 6445 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:31.089430 6447 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:59:31.098469 6447 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:59:31.104530 6447 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:31.104866 6447 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:59:31.108619 6447 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:59:31.113019 6447 catalog_manager.cc:6055] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Loaded TSK: 0
I20260430 19:59:31.114544 6447 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:59:31.118880 6447 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
W20260430 19:59:31.152549 6259 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:31.153657 6447 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260430 19:59:31.175360 5683 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_a4a99cd76d004be8a0e4b66d9e74fa77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:31.182982 5681 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_1f40269e3dc346979445899c9c93ac54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:31.180269 5684 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_5ef46544dc424dea9756db410f1a18c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:31.182243 5682 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:59:31.228060 5683 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:59:31.233512 5684 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5ef46544dc424dea9756db410f1a18c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:59:31.233695 5682 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260430 19:59:31.235375 5681 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1f40269e3dc346979445899c9c93ac54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260430 19:59:31.305364 6404 heartbeater.cc:499] Master 127.3.190.60:45507 was elected leader, sending a full tablet report...
W20260430 19:59:31.308030 6109 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 19:59:31.311635 6404 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:31.498569 5683 catalog_manager.cc:2174] created HMS catalog entry for table [id=f29979fa635848d98e8466f0a3576a6d]
W20260430 19:59:31.503517 5670 consensus_peers.cc:597] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f -> Peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Couldn't send request to peer 867d1f5512324df489edc5a61b79df54. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:59:31.524953 5768 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Refusing update from remote peer f46e27ec79e44f329de14da21f60e60f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260430 19:59:31.526780 6445 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260430 19:59:31.559767 5684 catalog_manager.cc:2174] created HMS catalog entry for table [id=e932eb2d012c43ff9471d92eb87f0bfb]
I20260430 19:59:31.565459 6452 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f46e27ec79e44f329de14da21f60e60f. Latest consensus state: current_term: 5 leader_uuid: "f46e27ec79e44f329de14da21f60e60f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:31.566208 6452 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:31.569506 6451 mvcc.cc:204] Tried to move back new op lower bound from 7280964286499946496 to 7280964284676235264. Current Snapshot: MvccSnapshot[applied={T|T < 7280964286499946496}]
I20260430 19:59:31.571440 6445 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "f46e27ec79e44f329de14da21f60e60f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:31.572158 6445 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:31.576586 5683 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52206:
table { table_name: "default.Table_a4a99cd76d004be8a0e4b66d9e74fa77" } new_table_name: "default.Table_8bf37998afb24f6db71c7e78a44337dc" modify_external_catalogs: true
I20260430 19:59:31.583791 6452 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "f46e27ec79e44f329de14da21f60e60f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:31.587235 6452 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:31.604846 5684 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52206:
table { table_name: "default.Table_5ef46544dc424dea9756db410f1a18c1" } new_table_name: "default.Table_83c2669f3f9548348584e0ecd6d59c9d" modify_external_catalogs: true
I20260430 19:59:31.689154 6193 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8b25a7d17492b8830605a61a2f2ea (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.691278 6194 tablet_service.cc:1511] Processing CreateTablet for tablet fb30716bb9174641ba41885f792e6364 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.692821 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb30716bb9174641ba41885f792e6364. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.699277 6193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8b25a7d17492b8830605a61a2f2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.712237 6043 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8b25a7d17492b8830605a61a2f2ea (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.715516 6044 tablet_service.cc:1511] Processing CreateTablet for tablet fb30716bb9174641ba41885f792e6364 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.716960 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb30716bb9174641ba41885f792e6364. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.724166 6043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8b25a7d17492b8830605a61a2f2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.737622 6338 tablet_service.cc:1511] Processing CreateTablet for tablet 8da8b25a7d17492b8830605a61a2f2ea (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.739282 6338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da8b25a7d17492b8830605a61a2f2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.736553 6339 tablet_service.cc:1511] Processing CreateTablet for tablet fb30716bb9174641ba41885f792e6364 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.749639 6464 tablet_bootstrap.cc:492] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:31.752167 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb30716bb9174641ba41885f792e6364. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.759768 6464 tablet_bootstrap.cc:654] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:31.762148 6464 log.cc:826] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf: Log is configured to *not* fsync() on all Append() calls
I20260430 19:59:31.765455 6465 tablet_bootstrap.cc:492] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:31.774315 6464 tablet_bootstrap.cc:492] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:31.774362 6465 tablet_bootstrap.cc:654] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:31.774889 6464 ts_tablet_manager.cc:1403] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.026s user 0.017s sys 0.003s
I20260430 19:59:31.777320 6465 log.cc:826] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Log is configured to *not* fsync() on all Append() calls
I20260430 19:59:31.777467 5680 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_955554de498d41cebb30bea37e154c53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:31.789729 6465 tablet_bootstrap.cc:492] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:31.790297 6465 ts_tablet_manager.cc:1403] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.005s
I20260430 19:59:31.791775 5679 master_service.cc:954] ReplaceTablet: received request to replace tablet fb30716bb9174641ba41885f792e6364 from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:31.792439 5678 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_c9b777e92e10430b8f8d1920f4ffc8bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:31.805400 5677 master_service.cc:954] ReplaceTablet: received request to replace tablet 8da8b25a7d17492b8830605a61a2f2ea from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:31.805884 6111 heartbeater.cc:499] Master 127.3.190.62:35659 was elected leader, sending a full tablet report...
I20260430 19:59:31.812522 6407 heartbeater.cc:499] Master 127.3.190.62:35659 was elected leader, sending a full tablet report...
I20260430 19:59:31.816468 5679 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet fb30716bb9174641ba41885f792e6364
I20260430 19:59:31.821157 5679 catalog_manager.cc:6741] ReplaceTablet: tablet fb30716bb9174641ba41885f792e6364 deleted and replaced by tablet 5972ed923bab43e7ac9a8d373109f1c9
I20260430 19:59:31.821486 6193 tablet_service.cc:1558] Processing DeleteTablet for tablet fb30716bb9174641ba41885f792e6364 with delete_type TABLET_DATA_DELETED (replaced by tablet 5972ed923bab43e7ac9a8d373109f1c9) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:31.823112 6337 tablet_service.cc:1558] Processing DeleteTablet for tablet fb30716bb9174641ba41885f792e6364 with delete_type TABLET_DATA_DELETED (replaced by tablet 5972ed923bab43e7ac9a8d373109f1c9) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:31.831377 6465 raft_consensus.cc:359] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:31.832297 6465 raft_consensus.cc:385] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:31.832597 6465 raft_consensus.cc:740] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:31.833564 6465 consensus_queue.cc:260] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:31.842330 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.60:45507
--webserver_interface=127.3.190.60
--webserver_port=37189
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--hive_metastore_uris=thrift://127.0.0.1:32957
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260430 19:59:31.843997 6465 ts_tablet_manager.cc:1434] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.053s user 0.036s sys 0.003s
I20260430 19:59:31.845008 6465 tablet_bootstrap.cc:492] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:31.851567 6465 tablet_bootstrap.cc:654] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
W20260430 19:59:31.861480 5670 catalog_manager.cc:4969] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet fb30716bb9174641ba41885f792e6364 because the tablet was not found. No further retry: Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:31.863976 6475 tablet_bootstrap.cc:492] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:31.866513 6465 tablet_bootstrap.cc:492] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:31.866990 6465 ts_tablet_manager.cc:1403] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.012s
I20260430 19:59:31.868624 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet fb30716bb9174641ba41885f792e6364 with delete_type TABLET_DATA_DELETED (replaced by tablet 5972ed923bab43e7ac9a8d373109f1c9) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:31.877282 6465 raft_consensus.cc:359] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:31.877998 6465 raft_consensus.cc:385] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:31.878273 6465 raft_consensus.cc:740] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:31.878937 6465 consensus_queue.cc:260] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:31.885565 6465 ts_tablet_manager.cc:1434] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20260430 19:59:31.887276 6476 tablet_replica.cc:333] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:31.888111 6476 raft_consensus.cc:2243] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:31.888633 6476 raft_consensus.cc:2272] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:31.891562 6476 ts_tablet_manager.cc:1916] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:31.894209 5677 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:31.899736 6475 tablet_bootstrap.cc:654] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:31.899657 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet 8da8b25a7d17492b8830605a61a2f2ea with delete_type TABLET_DATA_DELETED (replaced by tablet e053bb1db3b141df8699d5089781dfbb) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:31.901917 5677 catalog_manager.cc:6741] ReplaceTablet: tablet 8da8b25a7d17492b8830605a61a2f2ea deleted and replaced by tablet e053bb1db3b141df8699d5089781dfbb
I20260430 19:59:31.903513 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet 8da8b25a7d17492b8830605a61a2f2ea with delete_type TABLET_DATA_DELETED (replaced by tablet e053bb1db3b141df8699d5089781dfbb) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:31.908701 6476 ts_tablet_manager.cc:1929] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:31.909127 6476 log.cc:1199] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/fb30716bb9174641ba41885f792e6364
I20260430 19:59:31.910223 6476 ts_tablet_manager.cc:1950] T fb30716bb9174641ba41885f792e6364 P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:31.913590 6476 tablet_replica.cc:333] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:31.914280 5670 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet fb30716bb9174641ba41885f792e6364 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:31.902012 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet 8da8b25a7d17492b8830605a61a2f2ea with delete_type TABLET_DATA_DELETED (replaced by tablet e053bb1db3b141df8699d5089781dfbb) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:31.914799 6476 raft_consensus.cc:2243] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:31.915297 6476 raft_consensus.cc:2272] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260430 19:59:31.916366 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet 8da8b25a7d17492b8830605a61a2f2ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8da8b25a7d17492b8830605a61a2f2ea already in progress: creating tablet
I20260430 19:59:31.916914 6043 tablet_service.cc:1511] Processing CreateTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.917960 6043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5972ed923bab43e7ac9a8d373109f1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.918428 6194 tablet_service.cc:1511] Processing CreateTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.919485 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5972ed923bab43e7ac9a8d373109f1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.927970 6475 log.cc:826] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Log is configured to *not* fsync() on all Append() calls
I20260430 19:59:31.932246 6465 tablet_bootstrap.cc:492] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:31.934932 6476 ts_tablet_manager.cc:1916] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:31.940636 6465 tablet_bootstrap.cc:654] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:31.941386 6339 tablet_service.cc:1511] Processing CreateTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:31.942538 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5972ed923bab43e7ac9a8d373109f1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:31.949019 6476 ts_tablet_manager.cc:1929] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:31.949160 6465 tablet_bootstrap.cc:492] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:31.949426 6476 log.cc:1199] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:31.949622 6465 ts_tablet_manager.cc:1403] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.000s
I20260430 19:59:31.952288 6465 raft_consensus.cc:359] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:31.953876 6465 raft_consensus.cc:385] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:31.954170 6465 raft_consensus.cc:740] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:31.954389 6476 ts_tablet_manager.cc:1950] T 8da8b25a7d17492b8830605a61a2f2ea P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:31.957418 6475 tablet_bootstrap.cc:492] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:31.957954 6475 ts_tablet_manager.cc:1403] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.095s user 0.016s sys 0.000s
I20260430 19:59:31.961552 6465 consensus_queue.cc:260] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:31.966698 5670 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet 8da8b25a7d17492b8830605a61a2f2ea (table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]) successfully deleted
W20260430 19:59:31.968983 5671 catalog_manager.cc:4982] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet fb30716bb9174641ba41885f792e6364 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb30716bb9174641ba41885f792e6364 already in progress: creating tablet
I20260430 19:59:31.970638 6465 ts_tablet_manager.cc:1434] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
W20260430 19:59:31.979674 5671 catalog_manager.cc:4982] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet 8da8b25a7d17492b8830605a61a2f2ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8da8b25a7d17492b8830605a61a2f2ea already in progress: creating tablet
I20260430 19:59:31.990754 6475 raft_consensus.cc:359] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:31.991673 6475 raft_consensus.cc:385] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:31.991988 6475 raft_consensus.cc:740] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:31.993018 6475 consensus_queue.cc:260] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:32.008458 6475 ts_tablet_manager.cc:1434] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.050s user 0.018s sys 0.021s
I20260430 19:59:32.018147 6194 tablet_service.cc:1511] Processing CreateTablet for tablet e053bb1db3b141df8699d5089781dfbb (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.019337 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e053bb1db3b141df8699d5089781dfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.025104 6464 raft_consensus.cc:359] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.025966 6464 raft_consensus.cc:385] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.026264 6464 raft_consensus.cc:740] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.027251 6464 consensus_queue.cc:260] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
W20260430 19:59:32.035305 6411 tablet.cc:2404] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:59:32.036763 6475 tablet_bootstrap.cc:492] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:32.037098 5677 master_service.cc:954] ReplaceTablet: received request to replace tablet 5972ed923bab43e7ac9a8d373109f1c9 from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:32.052461 6475 tablet_bootstrap.cc:654] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
19:59:32.040 [HMS - ERROR - pool-6-thread-3] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_1f40269e3dc346979445899c9c93ac54 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
I20260430 19:59:32.060149 5677 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet 5972ed923bab43e7ac9a8d373109f1c9
I20260430 19:59:32.062423 6193 tablet_service.cc:1558] Processing DeleteTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1a877bc47c044cd0a5f4585ea702b0f1) from {username='slave'} at 127.0.0.1:44468
W20260430 19:59:32.063789 5681 catalog_manager.cc:2170] Already present: failed to create HMS catalog entry for table [id=ea4f904a659b4b979d6adb5d1dd636e9]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_1f40269e3dc346979445899c9c93ac54 already exists)
W20260430 19:59:32.064199 5671 catalog_manager.cc:4982] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet 5972ed923bab43e7ac9a8d373109f1c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5972ed923bab43e7ac9a8d373109f1c9 already in progress: creating tablet
I20260430 19:59:32.070075 6475 tablet_bootstrap.cc:492] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:32.069705 5681 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_8868e9393dc8483eb17d875dc93821ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:32.070520 6475 ts_tablet_manager.cc:1403] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.000s
I20260430 19:59:32.072124 5677 catalog_manager.cc:6741] ReplaceTablet: tablet 5972ed923bab43e7ac9a8d373109f1c9 deleted and replaced by tablet 1a877bc47c044cd0a5f4585ea702b0f1
I20260430 19:59:32.074494 6475 raft_consensus.cc:359] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.075182 6475 raft_consensus.cc:385] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.075461 6475 raft_consensus.cc:740] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.081746 6475 consensus_queue.cc:260] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.107474 6194 tablet_service.cc:1511] Processing CreateTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.108570 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a877bc47c044cd0a5f4585ea702b0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.112293 5677 master_service.cc:954] ReplaceTablet: received request to replace tablet 5972ed923bab43e7ac9a8d373109f1c9 from {username='slave'} at 127.0.0.1:52206
W20260430 19:59:32.120208 5670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.60:45507, user_credentials={real_user=slave}} blocked reactor thread for 97321.9us
I20260430 19:59:32.122996 6043 tablet_service.cc:1511] Processing CreateTablet for tablet e053bb1db3b141df8699d5089781dfbb (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.123307 6339 tablet_service.cc:1511] Processing CreateTablet for tablet e053bb1db3b141df8699d5089781dfbb (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.124145 6043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e053bb1db3b141df8699d5089781dfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.124325 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e053bb1db3b141df8699d5089781dfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.128342 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet fb30716bb9174641ba41885f792e6364 with delete_type TABLET_DATA_DELETED (replaced by tablet 5972ed923bab43e7ac9a8d373109f1c9) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.129261 6471 tablet_replica.cc:333] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
I20260430 19:59:32.130125 6471 raft_consensus.cc:2243] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.133378 6044 tablet_service.cc:1558] Processing DeleteTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1a877bc47c044cd0a5f4585ea702b0f1) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:32.134095 6476 tablet_replica.cc:333] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:32.134752 6476 raft_consensus.cc:2243] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.135226 6476 raft_consensus.cc:2272] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.137804 6476 ts_tablet_manager.cc:1916] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.139199 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1a877bc47c044cd0a5f4585ea702b0f1) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.140012 6044 tablet_service.cc:1511] Processing CreateTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.141111 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a877bc47c044cd0a5f4585ea702b0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.143903 6471 raft_consensus.cc:2272] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.146963 6471 ts_tablet_manager.cc:1916] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.147295 6263 heartbeater.cc:499] Master 127.3.190.62:35659 was elected leader, sending a full tablet report...
W20260430 19:59:32.152017 5677 catalog_manager.cc:5504] Ignoring report from unknown tablet fb30716bb9174641ba41885f792e6364
W20260430 19:59:32.152278 5677 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:32.156262 6338 tablet_service.cc:1511] Processing CreateTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.157372 6338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a877bc47c044cd0a5f4585ea702b0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.167485 6465 tablet_bootstrap.cc:492] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
W20260430 19:59:32.169117 5676 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:32.173223 6465 tablet_bootstrap.cc:654] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.177409 6193 tablet_service.cc:1558] Processing DeleteTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1a877bc47c044cd0a5f4585ea702b0f1) from {username='slave'} at 127.0.0.1:44468
W20260430 19:59:32.179068 5671 catalog_manager.cc:4982] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet 5972ed923bab43e7ac9a8d373109f1c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5972ed923bab43e7ac9a8d373109f1c9 already in progress: creating tablet
W20260430 19:59:32.184851 5677 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:32.188647 6464 ts_tablet_manager.cc:1434] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.413s user 0.050s sys 0.005s
I20260430 19:59:32.191927 6475 ts_tablet_manager.cc:1434] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.120s user 0.013s sys 0.001s
W20260430 19:59:32.200327 5677 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:32.200716 6476 ts_tablet_manager.cc:1929] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.201098 6476 log.cc:1199] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/5972ed923bab43e7ac9a8d373109f1c9
I20260430 19:59:32.201721 6476 ts_tablet_manager.cc:1950] T 5972ed923bab43e7ac9a8d373109f1c9 P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:32.202311 6475 tablet_bootstrap.cc:492] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:32.203945 6471 ts_tablet_manager.cc:1929] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.204357 6471 log.cc:1199] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/fb30716bb9174641ba41885f792e6364
I20260430 19:59:32.205641 6464 tablet_bootstrap.cc:492] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
W20260430 19:59:32.212669 5677 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:32.214959 6471 ts_tablet_manager.cc:1950] T fb30716bb9174641ba41885f792e6364 P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:32.217847 5670 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet fb30716bb9174641ba41885f792e6364 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.219082 6475 tablet_bootstrap.cc:654] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.219911 6465 tablet_bootstrap.cc:492] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:32.220359 6465 ts_tablet_manager.cc:1403] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.053s user 0.007s sys 0.018s
I20260430 19:59:32.223116 6465 raft_consensus.cc:359] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.223898 6465 raft_consensus.cc:385] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.224165 6465 raft_consensus.cc:740] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.224928 6465 consensus_queue.cc:260] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.227144 6465 ts_tablet_manager.cc:1434] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260430 19:59:32.230105 5670 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet 5972ed923bab43e7ac9a8d373109f1c9 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.230530 6465 tablet_bootstrap.cc:492] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
W20260430 19:59:32.233700 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet 5972ed923bab43e7ac9a8d373109f1c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5972ed923bab43e7ac9a8d373109f1c9 already in progress: creating tablet
I20260430 19:59:32.236632 6475 tablet_bootstrap.cc:492] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:32.237076 6475 ts_tablet_manager.cc:1403] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.005s
I20260430 19:59:32.241617 6475 raft_consensus.cc:359] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.242367 6475 raft_consensus.cc:385] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.242892 6475 raft_consensus.cc:740] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.243608 6475 consensus_queue.cc:260] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.244632 6464 tablet_bootstrap.cc:654] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.256592 6475 ts_tablet_manager.cc:1434] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.019s user 0.003s sys 0.003s
I20260430 19:59:32.259789 6465 tablet_bootstrap.cc:654] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.259734 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1a877bc47c044cd0a5f4585ea702b0f1) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.264839 6471 tablet_replica.cc:333] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
I20260430 19:59:32.265610 6471 raft_consensus.cc:2243] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.267215 6471 raft_consensus.cc:2272] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.269037 6471 ts_tablet_manager.cc:1916] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.279338 5677 master_service.cc:954] ReplaceTablet: received request to replace tablet 1a877bc47c044cd0a5f4585ea702b0f1 from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:32.283198 6465 tablet_bootstrap.cc:492] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:32.283654 6465 ts_tablet_manager.cc:1403] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.053s user 0.011s sys 0.000s
I20260430 19:59:32.286167 6465 raft_consensus.cc:359] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
I20260430 19:59:32.288710 6465 raft_consensus.cc:385] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.289047 6465 raft_consensus.cc:740] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.297200 6475 tablet_bootstrap.cc:492] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:32.298167 5677 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet 1a877bc47c044cd0a5f4585ea702b0f1
I20260430 19:59:32.289731 6465 consensus_queue.cc:260] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
I20260430 19:59:32.299597 5677 catalog_manager.cc:6741] ReplaceTablet: tablet 1a877bc47c044cd0a5f4585ea702b0f1 deleted and replaced by tablet da591d7d29cc463899ca44fcafd98e9f
I20260430 19:59:32.299914 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 with delete_type TABLET_DATA_DELETED (replaced by tablet da591d7d29cc463899ca44fcafd98e9f) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.301766 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 with delete_type TABLET_DATA_DELETED (replaced by tablet da591d7d29cc463899ca44fcafd98e9f) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.301870 6465 ts_tablet_manager.cc:1434] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.018s user 0.002s sys 0.004s
I20260430 19:59:32.303000 6044 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 with delete_type TABLET_DATA_DELETED (replaced by tablet da591d7d29cc463899ca44fcafd98e9f) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:32.303632 6476 tablet_replica.cc:333] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:32.304278 6476 raft_consensus.cc:2243] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.304723 6476 raft_consensus.cc:2272] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.306936 6476 ts_tablet_manager.cc:1916] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:59:32.306926 5671 catalog_manager.cc:4982] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet 1a877bc47c044cd0a5f4585ea702b0f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a877bc47c044cd0a5f4585ea702b0f1 already in progress: creating tablet
I20260430 19:59:32.311084 5682 catalog_manager.cc:2174] created HMS catalog entry for table [id=8bf2640796ce4d5081894bec93fd773d]
W20260430 19:59:32.315620 6267 tablet.cc:2404] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:59:32.318130 6475 tablet_bootstrap.cc:654] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.320230 5676 master_service.cc:954] ReplaceTablet: received request to replace tablet 1a877bc47c044cd0a5f4585ea702b0f1 from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:32.328069 6476 ts_tablet_manager.cc:1929] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.328423 6476 log.cc:1199] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/1a877bc47c044cd0a5f4585ea702b0f1
I20260430 19:59:32.329021 6476 ts_tablet_manager.cc:1950] T 1a877bc47c044cd0a5f4585ea702b0f1 P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:32.334764 6471 ts_tablet_manager.cc:1929] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.335193 6471 log.cc:1199] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/5972ed923bab43e7ac9a8d373109f1c9
I20260430 19:59:32.335860 6471 ts_tablet_manager.cc:1950] T 5972ed923bab43e7ac9a8d373109f1c9 P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:32.341749 5682 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52206 (request call id 14) took 1160 ms. Trace:
I20260430 19:59:32.342124 5682 rpcz_store.cc:276] 0430 19:59:31.181353 (+ 0us) service_pool.cc:168] Inserting onto call queue
0430 19:59:31.181572 (+ 219us) service_pool.cc:225] Handling call
0430 19:59:31.238731 (+ 57159us) catalog_manager.cc:2101] Acquired catalog manager lock
0430 19:59:31.240479 (+ 1748us) catalog_manager.cc:2147] Created new table and tablet info
0430 19:59:32.311044 (+1070565us) catalog_manager.cc:2173] Created new table in HMS catalog
0430 19:59:32.340989 (+ 29945us) catalog_manager.cc:2207] Wrote table and tablets to system table
0430 19:59:32.341352 (+ 363us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0430 19:59:32.341669 (+ 317us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0430 19:59:32.312825 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0430 19:59:32.313259 (+ 434us) write_op.cc:432] Acquiring schema lock in shared mode
0430 19:59:32.313283 (+ 24us) write_op.cc:435] Acquired schema lock
0430 19:59:32.313299 (+ 16us) tablet.cc:662] Decoding operations
0430 19:59:32.313588 (+ 289us) write_op.cc:620] Acquiring the partition lock for write op
0430 19:59:32.313632 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0430 19:59:32.313657 (+ 25us) tablet.cc:685] Acquiring locks for 2 operations
0430 19:59:32.313878 (+ 221us) tablet.cc:701] Row locks acquired
0430 19:59:32.313893 (+ 15us) write_op.cc:260] PREPARE: finished
0430 19:59:32.314015 (+ 122us) write_op.cc:270] Start()
0430 19:59:32.314100 (+ 85us) write_op.cc:276] Timestamp: P: 1777579172313973 usec, L: 0
0430 19:59:32.314119 (+ 19us) op_driver.cc:348] REPLICATION: starting
0430 19:59:32.314629 (+ 510us) log.cc:844] Serialized 432 byte log entry
0430 19:59:32.338150 (+ 23521us) op_driver.cc:464] REPLICATION: finished
0430 19:59:32.338682 (+ 532us) write_op.cc:301] APPLY: starting
0430 19:59:32.338743 (+ 61us) tablet.cc:1367] starting BulkCheckPresence
0430 19:59:32.338855 (+ 112us) tablet.cc:1370] finished BulkCheckPresence
0430 19:59:32.338936 (+ 81us) tablet.cc:1372] starting ApplyRowOperation cycle
0430 19:59:32.339296 (+ 360us) tablet.cc:1383] finished ApplyRowOperation cycle
0430 19:59:32.339445 (+ 149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0430 19:59:32.339489 (+ 44us) write_op.cc:312] APPLY: finished
0430 19:59:32.339680 (+ 191us) log.cc:844] Serialized 28 byte log entry
0430 19:59:32.339848 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:59:32.339919 (+ 71us) write_op.cc:454] Released schema lock
0430 19:59:32.339951 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0430 19:59:32.340176 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0430 19:59:32.340205 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":822069,"Hive Metastore.run_cpu_time_us":583,"Hive Metastore.run_wall_time_us":247887,"child_traces":[["op",{"apply.queue_time_us":324,"apply.run_cpu_time_us":1475,"apply.run_wall_time_us":1475,"num_ops":2,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":2124,"prepare.run_wall_time_us":2125,"raft.queue_time_us":190,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":15,"replication_time_us":23931}]]}
F20260430 19:59:32.346915 5956 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1777579171, eventType=CREATE_TABLE, dbName=default, tableName=table_a4a99cd76d004be8a0e4b66d9e74fa77, message={"server":"","servicePrincipal":"","db":"default","table":"table_a4a99cd76d004be8a0e4b66d9e74fa77","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a4a99cd76d004be8a0e4b66d9e74fa77\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777579171},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/warehouse/table_a4a99cd76d004be8a0e4b66d9e74fa77\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a4a99cd76d004be8a0e4b66d9e74fa77\",\"kudu.table_id\":\"f29979fa635848d98e8466f0a3576a6d\",\"transient_lastDdlTime\":\"1777579171\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4963134f6a694793a4115ae1c2aa919c\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777579171,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777579171, eventType=CREATE_TABLE, dbName=default, tableName=table_1f40269e3dc346979445899c9c93ac54, message={"server":"","servicePrincipal":"","db":"default","table":"table_1f40269e3dc346979445899c9c93ac54","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_1f40269e3dc346979445899c9c93ac54\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777579171},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/warehouse/table_1f40269e3dc346979445899c9c93ac54\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_1f40269e3dc346979445899c9c93ac54\",\"kudu.table_id\":\"7a6137ba6a5046ec945d466836f35e0a\",\"transient_lastDdlTime\":\"1777579171\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4963134f6a694793a4115ae1c2aa919c\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777579171,"files":[]}, messageFormat=json-0.2, catName=hive);
I20260430 19:59:32.355768 6475 tablet_bootstrap.cc:492] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:32.356237 6475 ts_tablet_manager.cc:1403] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.059s user 0.013s sys 0.000s
*** Check failure stack trace: ***
I20260430 19:59:32.359934 5670 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet 1a877bc47c044cd0a5f4585ea702b0f1 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.361002 5670 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet 5972ed923bab43e7ac9a8d373109f1c9 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.361479 5682 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52206:
table { table_name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7" } modify_external_catalogs: true
I20260430 19:59:32.362308 5682 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52206:
table { table_name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7" } modify_external_catalogs: true
I20260430 19:59:32.364276 6044 tablet_service.cc:1511] Processing CreateTablet for tablet da591d7d29cc463899ca44fcafd98e9f (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.365345 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da591d7d29cc463899ca44fcafd98e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.365568 6194 tablet_service.cc:1511] Processing CreateTablet for tablet da591d7d29cc463899ca44fcafd98e9f (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.366616 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da591d7d29cc463899ca44fcafd98e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.359572 6475 raft_consensus.cc:359] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
W20260430 19:59:32.369750 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet 1a877bc47c044cd0a5f4585ea702b0f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a877bc47c044cd0a5f4585ea702b0f1 already in progress: creating tablet
I20260430 19:59:32.370059 6475 raft_consensus.cc:385] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.370368 6475 raft_consensus.cc:740] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.371443 6475 consensus_queue.cc:260] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.380282 6339 tablet_service.cc:1511] Processing CreateTablet for tablet da591d7d29cc463899ca44fcafd98e9f (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.381349 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da591d7d29cc463899ca44fcafd98e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.385820 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 with delete_type TABLET_DATA_DELETED (replaced by tablet da591d7d29cc463899ca44fcafd98e9f) from {username='slave'} at 127.0.0.1:50868
W20260430 19:59:32.387280 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet 1a877bc47c044cd0a5f4585ea702b0f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1a877bc47c044cd0a5f4585ea702b0f1 already in progress: creating tablet
I20260430 19:59:32.388849 6475 ts_tablet_manager.cc:1434] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260430 19:59:32.366567 6464 tablet_bootstrap.cc:492] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:32.400398 6475 tablet_bootstrap.cc:492] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:32.400959 6464 ts_tablet_manager.cc:1403] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.195s user 0.014s sys 0.018s
I20260430 19:59:32.401934 6465 tablet_bootstrap.cc:492] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:32.403752 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 with delete_type TABLET_DATA_DELETED (replaced by tablet da591d7d29cc463899ca44fcafd98e9f) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.404459 6471 tablet_replica.cc:333] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
I20260430 19:59:32.405129 6471 raft_consensus.cc:2243] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.405575 6471 raft_consensus.cc:2272] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.407702 6465 tablet_bootstrap.cc:654] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.407663 6471 ts_tablet_manager.cc:1916] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.412045 6464 raft_consensus.cc:359] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:32.415019 6464 raft_consensus.cc:385] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.417217 6464 raft_consensus.cc:740] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.417994 6464 consensus_queue.cc:260] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:32.421080 6475 tablet_bootstrap.cc:654] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
@ 0x7fcc68c34950 google::LogMessage::Flush() at ??:0
I20260430 19:59:32.430932 6465 tablet_bootstrap.cc:492] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:32.431380 6465 ts_tablet_manager.cc:1403] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.000s
I20260430 19:59:32.431420 6464 ts_tablet_manager.cc:1434] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.029s user 0.003s sys 0.003s
I20260430 19:59:32.432777 6464 tablet_bootstrap.cc:492] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:32.433990 6465 raft_consensus.cc:359] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.434748 6465 raft_consensus.cc:385] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.435022 6465 raft_consensus.cc:740] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
W20260430 19:59:32.435945 5677 catalog_manager.cc:5504] Ignoring report from unknown tablet fb30716bb9174641ba41885f792e6364
W20260430 19:59:32.436286 6114 tablet.cc:2404] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260430 19:59:32.437536 6465 consensus_queue.cc:260] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.440040 6464 tablet_bootstrap.cc:654] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.452616 6465 ts_tablet_manager.cc:1434] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
W20260430 19:59:32.455439 5677 catalog_manager.cc:5504] Ignoring report from unknown tablet fb30716bb9174641ba41885f792e6364
I20260430 19:59:32.469328 6044 tablet_service.cc:1511] Processing CreateTablet for tablet 306a7c25190a4949ab0b1d80f447e684 (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.470425 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306a7c25190a4949ab0b1d80f447e684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.472627 6339 tablet_service.cc:1511] Processing CreateTablet for tablet 306a7c25190a4949ab0b1d80f447e684 (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.473717 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306a7c25190a4949ab0b1d80f447e684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.475790 6194 tablet_service.cc:1511] Processing CreateTablet for tablet 306a7c25190a4949ab0b1d80f447e684 (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.476833 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306a7c25190a4949ab0b1d80f447e684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.424355 6471 ts_tablet_manager.cc:1929] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.484304 6471 log.cc:1199] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/1a877bc47c044cd0a5f4585ea702b0f1
I20260430 19:59:32.484993 6471 ts_tablet_manager.cc:1950] T 1a877bc47c044cd0a5f4585ea702b0f1 P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:32.491953 5670 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet 1a877bc47c044cd0a5f4585ea702b0f1 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.497660 6481 raft_consensus.cc:493] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:32.498077 6481 raft_consensus.cc:515] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.500452 6481 leader_election.cc:290] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:32.501521 6465 tablet_bootstrap.cc:492] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
@ 0x7fcc68c346bf google::LogMessage::~LogMessage() at ??:0
I20260430 19:59:32.508013 6465 tablet_bootstrap.cc:654] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.528553 5677 master_service.cc:954] ReplaceTablet: received request to replace tablet 306a7c25190a4949ab0b1d80f447e684 from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:32.532536 5676 master_service.cc:954] ReplaceTablet: received request to replace tablet e053bb1db3b141df8699d5089781dfbb from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:32.536234 6472 raft_consensus.cc:493] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:32.536836 6472 raft_consensus.cc:515] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
I20260430 19:59:32.542610 6472 leader_election.cc:290] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:32.550556 6475 tablet_bootstrap.cc:492] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:32.551050 6475 ts_tablet_manager.cc:1403] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.151s user 0.011s sys 0.012s
I20260430 19:59:32.553951 6475 raft_consensus.cc:359] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
@ 0x7fcc73ceb2bc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260430 19:59:32.556887 6475 raft_consensus.cc:385] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.557214 6475 raft_consensus.cc:740] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.558732 6475 consensus_queue.cc:260] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
I20260430 19:59:32.559257 5675 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52206:
name: "default.Table_b2b7ae30e3404fe8be26c7cc54e7d102"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:32.578038 6475 ts_tablet_manager.cc:1434] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260430 19:59:32.582322 5677 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet 306a7c25190a4949ab0b1d80f447e684
I20260430 19:59:32.584014 6044 tablet_service.cc:1558] Processing DeleteTablet for tablet 306a7c25190a4949ab0b1d80f447e684 with delete_type TABLET_DATA_DELETED (replaced by tablet d28c8f73e1bd4acd8e30805e34662440) from {username='slave'} at 127.0.0.1:33948
W20260430 19:59:32.586151 5670 catalog_manager.cc:4982] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): delete failed for tablet 306a7c25190a4949ab0b1d80f447e684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 306a7c25190a4949ab0b1d80f447e684 already in progress: creating tablet
I20260430 19:59:32.586761 5677 catalog_manager.cc:6741] ReplaceTablet: tablet 306a7c25190a4949ab0b1d80f447e684 deleted and replaced by tablet d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:32.589581 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet 306a7c25190a4949ab0b1d80f447e684 with delete_type TABLET_DATA_DELETED (replaced by tablet d28c8f73e1bd4acd8e30805e34662440) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.591511 6193 tablet_service.cc:1558] Processing DeleteTablet for tablet 306a7c25190a4949ab0b1d80f447e684 with delete_type TABLET_DATA_DELETED (replaced by tablet d28c8f73e1bd4acd8e30805e34662440) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.591383 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8b25a7d17492b8830605a61a2f2ea" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
I20260430 19:59:32.592190 6359 raft_consensus.cc:2468] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
W20260430 19:59:32.594563 5671 catalog_manager.cc:4969] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet 306a7c25190a4949ab0b1d80f447e684 because the tablet was not found. No further retry: Not found: Tablet not found: 306a7c25190a4949ab0b1d80f447e684
I20260430 19:59:32.596302 6149 leader_election.cc:304] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf, cd13173a6c184acaa26b63bdbf8d6656; no voters:
I20260430 19:59:32.597195 6481 raft_consensus.cc:2804] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:32.597558 6481 raft_consensus.cc:493] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:32.597878 6481 raft_consensus.cc:3060] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:59:32.599079 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet 306a7c25190a4949ab0b1d80f447e684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 306a7c25190a4949ab0b1d80f447e684 already in progress: creating tablet
I20260430 19:59:32.602965 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e053bb1db3b141df8699d5089781dfbb" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
I20260430 19:59:32.603516 6359 raft_consensus.cc:2468] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 499a4336efe645d4926c94600c732b82 in term 0.
I20260430 19:59:32.605083 5999 leader_election.cc:304] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656; no voters:
I20260430 19:59:32.606532 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet e053bb1db3b141df8699d5089781dfbb with delete_type TABLET_DATA_DELETED (replaced by tablet fcb984dd833a443eac422b0821f4fa77) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.607184 6472 raft_consensus.cc:2804] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:32.607517 6472 raft_consensus.cc:493] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:32.607736 6471 tablet_replica.cc:333] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
I20260430 19:59:32.607837 6472 raft_consensus.cc:3060] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:32.607735 6481 raft_consensus.cc:515] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.608404 6471 raft_consensus.cc:2243] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.608837 6471 raft_consensus.cc:2272] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.610679 5676 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet e053bb1db3b141df8699d5089781dfbb
I20260430 19:59:32.612876 6471 ts_tablet_manager.cc:1916] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.615403 5676 catalog_manager.cc:6741] ReplaceTablet: tablet e053bb1db3b141df8699d5089781dfbb deleted and replaced by tablet fcb984dd833a443eac422b0821f4fa77
I20260430 19:59:32.617689 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da8b25a7d17492b8830605a61a2f2ea" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656"
I20260430 19:59:32.618250 6359 raft_consensus.cc:3060] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:32.618481 6044 tablet_service.cc:1558] Processing DeleteTablet for tablet e053bb1db3b141df8699d5089781dfbb with delete_type TABLET_DATA_DELETED (replaced by tablet fcb984dd833a443eac422b0821f4fa77) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:32.619199 6476 tablet_replica.cc:333] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:32.620185 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet e053bb1db3b141df8699d5089781dfbb with delete_type TABLET_DATA_DELETED (replaced by tablet fcb984dd833a443eac422b0821f4fa77) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.624770 6359 raft_consensus.cc:2468] T 8da8b25a7d17492b8830605a61a2f2ea P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:32.624934 6472 raft_consensus.cc:515] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
I20260430 19:59:32.625773 6476 raft_consensus.cc:2243] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.625785 6149 leader_election.cc:304] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf, cd13173a6c184acaa26b63bdbf8d6656; no voters:
I20260430 19:59:32.628214 6476 raft_consensus.cc:2272] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.630396 6472 leader_election.cc:290] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 election: Requested vote from peers 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:32.630870 6481 leader_election.cc:290] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Requested vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:32.631429 6481 raft_consensus.cc:2804] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:32.631861 6481 raft_consensus.cc:697] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [term 1 LEADER]: Becoming Leader. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Running, Role: LEADER
I20260430 19:59:32.632200 6464 tablet_bootstrap.cc:492] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:32.632735 6464 ts_tablet_manager.cc:1403] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.200s user 0.011s sys 0.007s
I20260430 19:59:32.632781 6481 consensus_queue.cc:237] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.635453 6476 ts_tablet_manager.cc:1916] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.636112 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e053bb1db3b141df8699d5089781dfbb" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656"
W20260430 19:59:32.637351 5999 leader_election.cc:343] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Illegal state: Tablet not RUNNING: STOPPED
I20260430 19:59:32.639940 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet e053bb1db3b141df8699d5089781dfbb with delete_type TABLET_DATA_DELETED (replaced by tablet fcb984dd833a443eac422b0821f4fa77) from {username='slave'} at 127.0.0.1:44468
W20260430 19:59:32.641623 5671 catalog_manager.cc:4982] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet e053bb1db3b141df8699d5089781dfbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e053bb1db3b141df8699d5089781dfbb already in progress: creating tablet
W20260430 19:59:32.651803 6149 leader_election.cc:343] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea
W20260430 19:59:32.652823 6149 leader_election.cc:343] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:32.655689 6044 tablet_service.cc:1511] Processing CreateTablet for tablet d28c8f73e1bd4acd8e30805e34662440 (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.656745 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d28c8f73e1bd4acd8e30805e34662440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.662706 6464 raft_consensus.cc:359] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.663381 6464 raft_consensus.cc:385] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.663631 6464 raft_consensus.cc:740] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.664325 6464 consensus_queue.cc:260] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
@ 0x7fcc73ceaa7c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260430 19:59:32.669800 6213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e053bb1db3b141df8699d5089781dfbb" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf"
I20260430 19:59:32.672857 6339 tablet_service.cc:1511] Processing CreateTablet for tablet d28c8f73e1bd4acd8e30805e34662440 (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.673209 6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e053bb1db3b141df8699d5089781dfbb" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf" is_pre_election: true
I20260430 19:59:32.673591 6476 ts_tablet_manager.cc:1929] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.673992 6476 log.cc:1199] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/e053bb1db3b141df8699d5089781dfbb
I20260430 19:59:32.674369 6194 tablet_service.cc:1511] Processing CreateTablet for tablet d28c8f73e1bd4acd8e30805e34662440 (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260430 19:59:32.674804 6000 leader_election.cc:343] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:59:32.675313 6000 leader_election.cc:304] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 499a4336efe645d4926c94600c732b82; no voters: 787275a4c5c64a6aa81be209814867bf, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:32.675521 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d28c8f73e1bd4acd8e30805e34662440. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:59:32.675972 6000 raft_consensus.cc:2686] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260430 19:59:32.676766 6000 leader_election.cc:343] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:59:32.677810 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d28c8f73e1bd4acd8e30805e34662440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.678808 6475 tablet_bootstrap.cc:492] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
W20260430 19:59:32.679855 5676 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:32.684963 6471 ts_tablet_manager.cc:1929] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.685359 6471 log.cc:1199] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/e053bb1db3b141df8699d5089781dfbb
I20260430 19:59:32.685760 6476 ts_tablet_manager.cc:1950] T e053bb1db3b141df8699d5089781dfbb P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:32.686072 6471 ts_tablet_manager.cc:1950] T e053bb1db3b141df8699d5089781dfbb P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:32.688421 5670 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet e053bb1db3b141df8699d5089781dfbb (table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]) successfully deleted
I20260430 19:59:32.689067 5670 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet e053bb1db3b141df8699d5089781dfbb (table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]) successfully deleted
I20260430 19:59:32.689603 6464 ts_tablet_manager.cc:1434] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.057s user 0.005s sys 0.001s
W20260430 19:59:32.689747 5670 catalog_manager.cc:4969] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet e053bb1db3b141df8699d5089781dfbb because the tablet was not found. No further retry: Not found: Tablet not found: e053bb1db3b141df8699d5089781dfbb
I20260430 19:59:32.690639 6464 tablet_bootstrap.cc:492] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:32.690702 6465 tablet_bootstrap.cc:492] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:32.691094 6465 ts_tablet_manager.cc:1403] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.190s user 0.022s sys 0.025s
I20260430 19:59:32.693487 6465 raft_consensus.cc:359] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.694073 6475 tablet_bootstrap.cc:654] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.694090 6465 raft_consensus.cc:385] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.694319 6465 raft_consensus.cc:740] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.694968 6465 consensus_queue.cc:260] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.696712 6465 ts_tablet_manager.cc:1434] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260430 19:59:32.702697 6193 tablet_service.cc:1558] Processing DeleteTablet for tablet 5972ed923bab43e7ac9a8d373109f1c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1a877bc47c044cd0a5f4585ea702b0f1) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.703559 6470 tablet_replica.cc:333] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:32.704406 6470 raft_consensus.cc:2243] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.704926 6470 raft_consensus.cc:2272] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.709905 6470 ts_tablet_manager.cc:1916] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.710729 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet 306a7c25190a4949ab0b1d80f447e684 with delete_type TABLET_DATA_DELETED (replaced by tablet d28c8f73e1bd4acd8e30805e34662440) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:32.711402 6476 tablet_replica.cc:333] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:32.712067 6476 raft_consensus.cc:2243] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.712497 6476 raft_consensus.cc:2272] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.714732 6476 ts_tablet_manager.cc:1916] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.722404 6464 tablet_bootstrap.cc:654] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.735297 6470 ts_tablet_manager.cc:1929] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.735718 6470 log.cc:1199] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/5972ed923bab43e7ac9a8d373109f1c9
I20260430 19:59:32.736521 6470 ts_tablet_manager.cc:1950] T 5972ed923bab43e7ac9a8d373109f1c9 P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:32.740307 6464 tablet_bootstrap.cc:492] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:32.740715 6464 ts_tablet_manager.cc:1403] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.050s user 0.010s sys 0.000s
I20260430 19:59:32.742164 6476 ts_tablet_manager.cc:1929] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.742522 6476 log.cc:1199] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/306a7c25190a4949ab0b1d80f447e684
I20260430 19:59:32.743093 6465 tablet_bootstrap.cc:492] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:32.743196 6476 ts_tablet_manager.cc:1950] T 306a7c25190a4949ab0b1d80f447e684 P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:32.747026 6464 raft_consensus.cc:359] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
I20260430 19:59:32.747749 6464 raft_consensus.cc:385] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
@ 0x7fcc73ceeaf2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260430 19:59:32.749527 6465 tablet_bootstrap.cc:654] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.752278 5671 catalog_manager.cc:5002] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): tablet 5972ed923bab43e7ac9a8d373109f1c9 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.748102 6464 raft_consensus.cc:740] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.755455 6475 tablet_bootstrap.cc:492] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:32.755905 6475 ts_tablet_manager.cc:1403] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.077s user 0.014s sys 0.005s
I20260430 19:59:32.759933 6464 consensus_queue.cc:260] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } }
I20260430 19:59:32.768288 5670 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet 306a7c25190a4949ab0b1d80f447e684 (table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]) successfully deleted
I20260430 19:59:32.768654 6464 ts_tablet_manager.cc:1434] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260430 19:59:32.769589 6464 tablet_bootstrap.cc:492] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:32.758520 6475 raft_consensus.cc:359] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.770819 6475 raft_consensus.cc:385] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.771242 6475 raft_consensus.cc:740] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.772161 6475 consensus_queue.cc:260] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.782851 6044 tablet_service.cc:1511] Processing CreateTablet for tablet fcb984dd833a443eac422b0821f4fa77 (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.783983 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb984dd833a443eac422b0821f4fa77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.786037 6193 tablet_service.cc:1511] Processing CreateTablet for tablet fcb984dd833a443eac422b0821f4fa77 (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.787127 6193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb984dd833a443eac422b0821f4fa77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.787317 6339 tablet_service.cc:1511] Processing CreateTablet for tablet fcb984dd833a443eac422b0821f4fa77 (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.789965 6475 ts_tablet_manager.cc:1434] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.034s user 0.007s sys 0.000s
I20260430 19:59:32.792603 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb984dd833a443eac422b0821f4fa77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.793748 5679 master_service.cc:954] ReplaceTablet: received request to replace tablet da591d7d29cc463899ca44fcafd98e9f from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:32.807112 6475 tablet_bootstrap.cc:492] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:32.812657 5679 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet da591d7d29cc463899ca44fcafd98e9f
I20260430 19:59:32.814389 5679 catalog_manager.cc:6741] ReplaceTablet: tablet da591d7d29cc463899ca44fcafd98e9f deleted and replaced by tablet 69deeb4a53f04b1b87be5d406890d10b
I20260430 19:59:32.814363 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet da591d7d29cc463899ca44fcafd98e9f with delete_type TABLET_DATA_DELETED (replaced by tablet 69deeb4a53f04b1b87be5d406890d10b) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:32.814993 6192 tablet_service.cc:1558] Processing DeleteTablet for tablet da591d7d29cc463899ca44fcafd98e9f with delete_type TABLET_DATA_DELETED (replaced by tablet 69deeb4a53f04b1b87be5d406890d10b) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.815423 6476 tablet_replica.cc:333] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:32.816056 6476 raft_consensus.cc:2243] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.816500 6476 raft_consensus.cc:2272] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260430 19:59:32.816612 5671 catalog_manager.cc:4982] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet da591d7d29cc463899ca44fcafd98e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da591d7d29cc463899ca44fcafd98e9f already in progress: creating tablet
I20260430 19:59:32.817204 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet da591d7d29cc463899ca44fcafd98e9f with delete_type TABLET_DATA_DELETED (replaced by tablet 69deeb4a53f04b1b87be5d406890d10b) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.817940 6471 tablet_replica.cc:333] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
I20260430 19:59:32.818694 6471 raft_consensus.cc:2243] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.819180 6476 ts_tablet_manager.cc:1916] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.819176 6471 raft_consensus.cc:2272] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.820690 6481 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:32.821182 6471 ts_tablet_manager.cc:1916] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.821115 6481 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:32.822887 6481 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 19:59:32.827134 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 19:59:32.828037 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:32.828403 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:32.829408 6481 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
@ 0x7fcc73ceeaaa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260430 19:59:32.832250 5677 master_service.cc:954] ReplaceTablet: received request to replace tablet fcb984dd833a443eac422b0821f4fa77 from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:32.834969 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet e053bb1db3b141df8699d5089781dfbb with delete_type TABLET_DATA_DELETED (replaced by tablet fcb984dd833a443eac422b0821f4fa77) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.835793 6470 tablet_replica.cc:333] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:32.836475 6470 raft_consensus.cc:2243] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.836931 6470 raft_consensus.cc:2272] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.839170 6470 ts_tablet_manager.cc:1916] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.840723 6476 ts_tablet_manager.cc:1929] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.841054 6476 log.cc:1199] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/da591d7d29cc463899ca44fcafd98e9f
I20260430 19:59:32.841617 6476 ts_tablet_manager.cc:1950] T da591d7d29cc463899ca44fcafd98e9f P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:32.843957 5670 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet da591d7d29cc463899ca44fcafd98e9f (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.845695 6464 tablet_bootstrap.cc:654] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.852648 6465 tablet_bootstrap.cc:492] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:32.853084 6465 ts_tablet_manager.cc:1403] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.110s user 0.012s sys 0.000s
@ 0x7fcc73ceea3a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260430 19:59:32.855440 6465 raft_consensus.cc:359] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.856024 6465 raft_consensus.cc:385] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.856256 6465 raft_consensus.cc:740] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.857509 6475 tablet_bootstrap.cc:654] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.863612 6464 tablet_bootstrap.cc:492] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:32.864039 6464 ts_tablet_manager.cc:1403] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.095s user 0.010s sys 0.005s
I20260430 19:59:32.866582 6464 raft_consensus.cc:359] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.870937 6464 raft_consensus.cc:385] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.871256 6464 raft_consensus.cc:740] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.871937 6464 consensus_queue.cc:260] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.876685 6471 ts_tablet_manager.cc:1929] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:32.877064 6471 log.cc:1199] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/da591d7d29cc463899ca44fcafd98e9f
I20260430 19:59:32.877416 6464 ts_tablet_manager.cc:1434] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260430 19:59:32.877719 6471 ts_tablet_manager.cc:1950] T da591d7d29cc463899ca44fcafd98e9f P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:32.878233 6464 tablet_bootstrap.cc:492] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:32.880239 5670 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet da591d7d29cc463899ca44fcafd98e9f (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:32.886117 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a877bc47c044cd0a5f4585ea702b0f1 with delete_type TABLET_DATA_DELETED (replaced by tablet da591d7d29cc463899ca44fcafd98e9f) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.887113 6044 tablet_service.cc:1511] Processing CreateTablet for tablet 69deeb4a53f04b1b87be5d406890d10b (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.888198 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69deeb4a53f04b1b87be5d406890d10b. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fcc73ceea02 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260430 19:59:32.901968 6464 tablet_bootstrap.cc:654] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.904770 5677 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet fcb984dd833a443eac422b0821f4fa77
I20260430 19:59:32.906138 5677 catalog_manager.cc:6741] ReplaceTablet: tablet fcb984dd833a443eac422b0821f4fa77 deleted and replaced by tablet b26883f75c5b4b3480bbc43cfd03e1ee
I20260430 19:59:32.907999 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet fcb984dd833a443eac422b0821f4fa77 with delete_type TABLET_DATA_DELETED (replaced by tablet b26883f75c5b4b3480bbc43cfd03e1ee) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.908957 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet fcb984dd833a443eac422b0821f4fa77 with delete_type TABLET_DATA_DELETED (replaced by tablet b26883f75c5b4b3480bbc43cfd03e1ee) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.909591 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet fcb984dd833a443eac422b0821f4fa77 with delete_type TABLET_DATA_DELETED (replaced by tablet b26883f75c5b4b3480bbc43cfd03e1ee) from {username='slave'} at 127.0.0.1:33948
W20260430 19:59:32.910475 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet fcb984dd833a443eac422b0821f4fa77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcb984dd833a443eac422b0821f4fa77 already in progress: creating tablet
W20260430 19:59:32.911217 5670 catalog_manager.cc:4982] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): delete failed for tablet fcb984dd833a443eac422b0821f4fa77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fcb984dd833a443eac422b0821f4fa77 already in progress: creating tablet
I20260430 19:59:32.913100 6465 consensus_queue.cc:260] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
@ 0x7fcc73cedcfe std::__1::__function::__func<>::operator()() at ??:0
@ 0x5b3bc5 std::__1::__function::__value_func<>::operator()() at ??:?
I20260430 19:59:32.917935 6194 tablet_service.cc:1511] Processing CreateTablet for tablet 69deeb4a53f04b1b87be5d406890d10b (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.919389 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69deeb4a53f04b1b87be5d406890d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.925015 6465 ts_tablet_manager.cc:1434] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.072s user 0.006s sys 0.000s
I20260430 19:59:32.925913 6465 tablet_bootstrap.cc:492] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:32.927183 6464 tablet_bootstrap.cc:492] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:32.927596 6464 ts_tablet_manager.cc:1403] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.003s
@ 0x5b39fa std::__1::function<>::operator()() at ??:?
I20260430 19:59:32.941181 6339 tablet_service.cc:1511] Processing CreateTablet for tablet 69deeb4a53f04b1b87be5d406890d10b (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:32.942226 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69deeb4a53f04b1b87be5d406890d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:32.930222 6464 raft_consensus.cc:359] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.942734 6464 raft_consensus.cc:385] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.943064 6464 raft_consensus.cc:740] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.943768 6464 consensus_queue.cc:260] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.952128 6464 ts_tablet_manager.cc:1434] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260430 19:59:32.954782 6475 tablet_bootstrap.cc:492] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:32.955183 6475 ts_tablet_manager.cc:1403] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.148s user 0.011s sys 0.023s
I20260430 19:59:32.955299 6464 tablet_bootstrap.cc:492] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:32.957580 6475 raft_consensus.cc:359] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.958190 6475 raft_consensus.cc:385] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:32.958420 6475 raft_consensus.cc:740] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:32.961514 6470 ts_tablet_manager.cc:1929] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
@ 0x7fcc6ae5afbb kudu::Thread::SuperviseThread() at ??:0
@ 0x479510 __tsan_thread_start_func at ??:?
I20260430 19:59:32.969959 6475 consensus_queue.cc:260] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.967792 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet da591d7d29cc463899ca44fcafd98e9f with delete_type TABLET_DATA_DELETED (replaced by tablet 69deeb4a53f04b1b87be5d406890d10b) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:32.961890 6470 log.cc:1199] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/e053bb1db3b141df8699d5089781dfbb
I20260430 19:59:32.972867 6470 ts_tablet_manager.cc:1950] T e053bb1db3b141df8699d5089781dfbb P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:32.961892 6464 tablet_bootstrap.cc:654] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.978396 6465 tablet_bootstrap.cc:654] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:32.979885 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet 306a7c25190a4949ab0b1d80f447e684 with delete_type TABLET_DATA_DELETED (replaced by tablet d28c8f73e1bd4acd8e30805e34662440) from {username='slave'} at 127.0.0.1:50868
W20260430 19:59:32.981686 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet 306a7c25190a4949ab0b1d80f447e684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 306a7c25190a4949ab0b1d80f447e684 already in progress: creating tablet
I20260430 19:59:32.984642 6475 ts_tablet_manager.cc:1434] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260430 19:59:32.985737 5671 catalog_manager.cc:5002] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): tablet e053bb1db3b141df8699d5089781dfbb (table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]) successfully deleted
I20260430 19:59:32.986713 6470 tablet_replica.cc:333] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:32.988911 6481 raft_consensus.cc:493] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:32.989346 6481 raft_consensus.cc:515] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:32.989295 6470 raft_consensus.cc:2243] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.990520 6470 raft_consensus.cc:2272] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.991199 6481 leader_election.cc:290] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:32.994153 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet 306a7c25190a4949ab0b1d80f447e684 with delete_type TABLET_DATA_DELETED (replaced by tablet d28c8f73e1bd4acd8e30805e34662440) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:32.994813 6471 tablet_replica.cc:333] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
W20260430 19:59:32.994859 6149 leader_election.cc:343] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: da591d7d29cc463899ca44fcafd98e9f
W20260430 19:59:32.995440 6149 leader_election.cc:343] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: da591d7d29cc463899ca44fcafd98e9f
I20260430 19:59:32.995492 6471 raft_consensus.cc:2243] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:32.995765 6149 leader_election.cc:304] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:32.995931 6471 raft_consensus.cc:2272] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:32.996847 6481 raft_consensus.cc:2749] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:32.996955 6475 tablet_bootstrap.cc:492] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:32.998708 6471 ts_tablet_manager.cc:1916] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:32.998857 6470 ts_tablet_manager.cc:1916] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:33.003957 6475 tablet_bootstrap.cc:654] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.008607 6464 tablet_bootstrap.cc:492] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:33.009006 6464 ts_tablet_manager.cc:1403] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.004s
I20260430 19:59:33.011287 6464 raft_consensus.cc:359] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.011852 6464 raft_consensus.cc:385] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.012099 6464 raft_consensus.cc:740] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.013676 6464 consensus_queue.cc:260] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.014189 6044 tablet_service.cc:1511] Processing CreateTablet for tablet b26883f75c5b4b3480bbc43cfd03e1ee (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:33.015272 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b26883f75c5b4b3480bbc43cfd03e1ee. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fcc6c54d6db start_thread at ??:0
I20260430 19:59:33.021347 5677 master_service.cc:954] ReplaceTablet: received request to replace tablet d28c8f73e1bd4acd8e30805e34662440 from {username='slave'} at 127.0.0.1:52206
I20260430 19:59:33.025929 6471 ts_tablet_manager.cc:1929] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:33.026350 6471 log.cc:1199] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/306a7c25190a4949ab0b1d80f447e684
I20260430 19:59:33.027065 6471 ts_tablet_manager.cc:1950] T 306a7c25190a4949ab0b1d80f447e684 P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:33.029558 5670 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet 306a7c25190a4949ab0b1d80f447e684 (table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]) successfully deleted
I20260430 19:59:33.031319 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet 306a7c25190a4949ab0b1d80f447e684 with delete_type TABLET_DATA_DELETED (replaced by tablet d28c8f73e1bd4acd8e30805e34662440) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:33.033376 6464 ts_tablet_manager.cc:1434] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260430 19:59:33.046711 6464 tablet_bootstrap.cc:492] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:33.048669 5677 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f46e27ec79e44f329de14da21f60e60f: Sending DeleteTablet for 3 replicas of tablet d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:33.050453 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet d28c8f73e1bd4acd8e30805e34662440 with delete_type TABLET_DATA_DELETED (replaced by tablet 87720d2b47d6419bb1420c6f0187d74b) from {username='slave'} at 127.0.0.1:33948
I20260430 19:59:33.051231 6476 tablet_replica.cc:333] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:33.051784 6476 raft_consensus.cc:2243] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:33.052212 6476 raft_consensus.cc:2272] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:33.055011 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet d28c8f73e1bd4acd8e30805e34662440 with delete_type TABLET_DATA_DELETED (replaced by tablet 87720d2b47d6419bb1420c6f0187d74b) from {username='slave'} at 127.0.0.1:50868
I20260430 19:59:33.055519 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet d28c8f73e1bd4acd8e30805e34662440 with delete_type TABLET_DATA_DELETED (replaced by tablet 87720d2b47d6419bb1420c6f0187d74b) from {username='slave'} at 127.0.0.1:44468
I20260430 19:59:33.056351 6476 ts_tablet_manager.cc:1916] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:59:33.056593 5670 catalog_manager.cc:4982] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet d28c8f73e1bd4acd8e30805e34662440 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d28c8f73e1bd4acd8e30805e34662440 already in progress: creating tablet
I20260430 19:59:33.058820 5677 catalog_manager.cc:6741] ReplaceTablet: tablet d28c8f73e1bd4acd8e30805e34662440 deleted and replaced by tablet 87720d2b47d6419bb1420c6f0187d74b
@ 0x7fcc678d961f clone at ??:0
I20260430 19:59:33.052232 6464 tablet_bootstrap.cc:654] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.066303 6470 ts_tablet_manager.cc:1929] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:33.071621 6470 log.cc:1199] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/1a877bc47c044cd0a5f4585ea702b0f1
I20260430 19:59:33.072275 6470 ts_tablet_manager.cc:1950] T 1a877bc47c044cd0a5f4585ea702b0f1 P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:33.080272 6465 tablet_bootstrap.cc:492] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:33.080683 6465 ts_tablet_manager.cc:1403] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.155s user 0.008s sys 0.000s
I20260430 19:59:33.080762 6470 tablet_replica.cc:333] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:33.082314 6470 raft_consensus.cc:2243] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:33.082814 6470 raft_consensus.cc:2272] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:33.086875 6470 ts_tablet_manager.cc:1916] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:33.086902 6465 raft_consensus.cc:359] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.087489 6465 raft_consensus.cc:385] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.087725 6465 raft_consensus.cc:740] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.090402 6465 consensus_queue.cc:260] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.094285 6464 tablet_bootstrap.cc:492] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:33.094702 6464 ts_tablet_manager.cc:1403] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.004s
I20260430 19:59:33.097097 6464 raft_consensus.cc:359] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.098085 6464 raft_consensus.cc:385] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.098872 6476 ts_tablet_manager.cc:1929] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:33.099040 6464 raft_consensus.cc:740] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.099206 6476 log.cc:1199] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:33.099807 6476 ts_tablet_manager.cc:1950] T d28c8f73e1bd4acd8e30805e34662440 P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:33.100553 6464 consensus_queue.cc:260] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.102325 6465 ts_tablet_manager.cc:1434] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260430 19:59:33.108368 6464 ts_tablet_manager.cc:1434] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.013s user 0.004s sys 0.001s
I20260430 19:59:33.109252 6464 tablet_bootstrap.cc:492] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:33.123095 6464 tablet_bootstrap.cc:654] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.123145 6465 tablet_bootstrap.cc:492] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:33.129966 6470 ts_tablet_manager.cc:1929] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:33.130349 6470 log.cc:1199] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/da591d7d29cc463899ca44fcafd98e9f
I20260430 19:59:33.131013 6470 ts_tablet_manager.cc:1950] T da591d7d29cc463899ca44fcafd98e9f P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:33.133260 6475 tablet_bootstrap.cc:492] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:33.133669 6475 ts_tablet_manager.cc:1403] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.137s user 0.030s sys 0.050s
I20260430 19:59:33.135519 6464 tablet_bootstrap.cc:492] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:33.135900 6464 ts_tablet_manager.cc:1403] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.001s
I20260430 19:59:33.135903 6475 raft_consensus.cc:359] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.136497 6475 raft_consensus.cc:385] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.137673 6475 raft_consensus.cc:740] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.138182 6464 raft_consensus.cc:359] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.138270 6475 consensus_queue.cc:260] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.138844 6464 raft_consensus.cc:385] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.139081 6464 raft_consensus.cc:740] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.140029 6475 ts_tablet_manager.cc:1434] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260430 19:59:33.140861 6475 tablet_bootstrap.cc:492] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:33.141157 6464 consensus_queue.cc:260] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.143137 6481 raft_consensus.cc:493] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:33.143529 6481 raft_consensus.cc:515] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.145717 6465 tablet_bootstrap.cc:654] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.148036 6064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb984dd833a443eac422b0821f4fa77" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:33.148746 6064 raft_consensus.cc:2468] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:33.149998 6149 leader_election.cc:304] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:33.150938 6500 raft_consensus.cc:2804] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:33.151474 6500 raft_consensus.cc:493] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:33.151813 6500 raft_consensus.cc:3060] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:33.153784 6475 tablet_bootstrap.cc:654] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.155135 6481 leader_election.cc:290] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:33.156839 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb984dd833a443eac422b0821f4fa77" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
W20260430 19:59:33.158278 6149 leader_election.cc:343] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:59:33.163336 6500 raft_consensus.cc:515] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.166136 6064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb984dd833a443eac422b0821f4fa77" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82"
I20260430 19:59:33.166726 6064 raft_consensus.cc:3060] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:33.173581 6064 raft_consensus.cc:2468] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:33.174804 6149 leader_election.cc:304] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:33.175669 6481 raft_consensus.cc:2804] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:33.177160 6470 tablet_replica.cc:333] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:33.177911 6470 raft_consensus.cc:2243] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:33.178377 6470 raft_consensus.cc:2272] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:33.181155 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb984dd833a443eac422b0821f4fa77" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656"
W20260430 19:59:33.182715 6149 leader_election.cc:343] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:59:33.183790 6500 leader_election.cc:290] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Requested vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:33.187628 6470 ts_tablet_manager.cc:1916] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:33.192943 6481 raft_consensus.cc:697] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [term 1 LEADER]: Becoming Leader. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Running, Role: LEADER
I20260430 19:59:33.193819 6481 consensus_queue.cc:237] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.201083 6465 tablet_bootstrap.cc:492] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:33.202302 6465 ts_tablet_manager.cc:1403] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.079s user 0.021s sys 0.011s
I20260430 19:59:33.204944 6465 raft_consensus.cc:359] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.205667 6465 raft_consensus.cc:385] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.206153 6465 raft_consensus.cc:740] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.207049 6465 consensus_queue.cc:260] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.211364 6464 ts_tablet_manager.cc:1434] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.075s user 0.008s sys 0.000s
I20260430 19:59:33.214228 6465 ts_tablet_manager.cc:1434] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260430 19:59:33.215126 6465 tablet_bootstrap.cc:492] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:33.215782 6483 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:33.216272 6483 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.218835 6464 tablet_bootstrap.cc:492] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:33.225117 6464 tablet_bootstrap.cc:654] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.225306 6475 tablet_bootstrap.cc:492] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:33.225689 6475 ts_tablet_manager.cc:1403] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.085s user 0.014s sys 0.017s
I20260430 19:59:33.228045 6475 raft_consensus.cc:359] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.228622 6475 raft_consensus.cc:385] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.228856 6475 raft_consensus.cc:740] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.229521 6475 consensus_queue.cc:260] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.239526 6464 tablet_bootstrap.cc:492] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:33.239969 6464 ts_tablet_manager.cc:1403] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.000s
I20260430 19:59:33.242058 6465 tablet_bootstrap.cc:654] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.243404 6464 raft_consensus.cc:359] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.244019 6464 raft_consensus.cc:385] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.244257 6464 raft_consensus.cc:740] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.244799 6464 consensus_queue.cc:260] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.248725 6470 ts_tablet_manager.cc:1929] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:33.249110 6470 log.cc:1199] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/306a7c25190a4949ab0b1d80f447e684
I20260430 19:59:33.249728 6470 ts_tablet_manager.cc:1950] T 306a7c25190a4949ab0b1d80f447e684 P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:33.252239 6483 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:33.254300 6464 ts_tablet_manager.cc:1434] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260430 19:59:33.258719 6465 tablet_bootstrap.cc:492] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:33.259135 6465 ts_tablet_manager.cc:1403] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.002s
I20260430 19:59:33.271867 6475 ts_tablet_manager.cc:1434] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.046s user 0.007s sys 0.002s
I20260430 19:59:33.272809 6475 tablet_bootstrap.cc:492] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:33.272989 6470 tablet_replica.cc:333] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:33.273686 6470 raft_consensus.cc:2243] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:33.274161 6470 raft_consensus.cc:2272] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:33.277909 6475 tablet_bootstrap.cc:654] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:33.279271 6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d28c8f73e1bd4acd8e30805e34662440" candidate_uuid: "cd13173a6c184acaa26b63bdbf8d6656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf" is_pre_election: true
W20260430 19:59:33.280561 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:59:33.280534 6465 raft_consensus.cc:359] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.281147 6465 raft_consensus.cc:385] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.281389 6465 raft_consensus.cc:740] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.282228 6470 ts_tablet_manager.cc:1916] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260430 19:59:33.288861 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:33.289280 6294 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 19:59:33.291364 6532 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:33.288017 6465 consensus_queue.cc:260] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.296408 6465 ts_tablet_manager.cc:1434] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260430 19:59:33.300443 6470 ts_tablet_manager.cc:1929] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:33.300817 6470 log.cc:1199] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:33.301432 6470 ts_tablet_manager.cc:1950] T d28c8f73e1bd4acd8e30805e34662440 P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:33.310050 6475 tablet_bootstrap.cc:492] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:33.310496 6475 ts_tablet_manager.cc:1403] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.038s user 0.011s sys 0.003s
I20260430 19:59:33.314287 6475 raft_consensus.cc:359] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.315024 6475 raft_consensus.cc:385] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:33.315287 6475 raft_consensus.cc:740] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:33.316030 6475 consensus_queue.cc:260] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.320734 6475 ts_tablet_manager.cc:1434] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260430 19:59:33.336699 6472 raft_consensus.cc:493] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:33.337160 6472 raft_consensus.cc:515] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.339043 6472 leader_election.cc:290] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 19:59:33.341348 6000 leader_election.cc:343] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: b26883f75c5b4b3480bbc43cfd03e1ee
W20260430 19:59:33.341988 5999 leader_election.cc:343] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: b26883f75c5b4b3480bbc43cfd03e1ee
I20260430 19:59:33.342350 5999 leader_election.cc:304] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 499a4336efe645d4926c94600c732b82; no voters: 787275a4c5c64a6aa81be209814867bf, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:33.343286 6472 raft_consensus.cc:2749] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 19:59:33.344002 6473 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:33.346923 6473 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:33.347615 6473 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260430 19:59:33.348755 6500 raft_consensus.cc:493] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:33.349195 6500 raft_consensus.cc:515] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.350940 6500 leader_election.cc:290] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:33.352576 6064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69deeb4a53f04b1b87be5d406890d10b" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:33.352607 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69deeb4a53f04b1b87be5d406890d10b" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
I20260430 19:59:33.353132 6064 raft_consensus.cc:2468] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:33.353165 6359 raft_consensus.cc:2468] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:33.354179 6149 leader_election.cc:304] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf, cd13173a6c184acaa26b63bdbf8d6656; no voters:
I20260430 19:59:33.355063 6500 raft_consensus.cc:2804] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:33.355366 6500 raft_consensus.cc:493] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:33.355638 6500 raft_consensus.cc:3060] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:33.362506 6500 raft_consensus.cc:515] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.365147 6064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69deeb4a53f04b1b87be5d406890d10b" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82"
I20260430 19:59:33.365670 6064 raft_consensus.cc:3060] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:33.371770 6064 raft_consensus.cc:2468] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:33.373024 6149 leader_election.cc:304] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:33.373910 6481 raft_consensus.cc:2804] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:33.374338 6481 raft_consensus.cc:697] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 1 LEADER]: Becoming Leader. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Running, Role: LEADER
I20260430 19:59:33.375167 6481 consensus_queue.cc:237] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.382030 6500 leader_election.cc:290] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Requested vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:33.389016 6359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69deeb4a53f04b1b87be5d406890d10b" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656"
I20260430 19:59:33.389593 6359 raft_consensus.cc:3060] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:33.395733 6359 raft_consensus.cc:2468] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
W20260430 19:59:33.426568 6473 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260430 19:59:33.426967 6473 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260430 19:59:33.427229 6473 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:59:33.427435 6473 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:33.427629 6473 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:59:33.427824 6473 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:59:33.476775 6473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:32957
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.60:45507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.190.60
--webserver_port=37189
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:33.478137 6473 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:33.479840 6473 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:33.481334 6147 connection.cc:570] server connection from 127.0.0.1:44468 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:33.481359 6292 connection.cc:570] client connection to 127.3.190.62:35659 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:33.481912 6147 connection.cc:570] client connection to 127.3.190.62:35659 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:33.482951 6414 connection.cc:570] client connection to 127.3.190.62:35659 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:33.483412 5999 connection.cc:570] server connection from 127.0.0.1:33948 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:33.484156 5997 connection.cc:570] client connection to 127.3.190.62:35659 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:33.485195 6414 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.190.62:35659)
W20260430 19:59:33.485896 6414 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.190.62:35659)
W20260430 19:59:33.490088 6414 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.190.62:35659)
W20260430 19:59:33.493489 6414 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.190.62:35659)
W20260430 19:59:33.498183 6545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:33.565660 6552 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f46e27ec79e44f329de14da21f60e60f)
I20260430 19:59:33.566514 6552 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:33.583575 6552 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:33.597457 5742 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:33.598384 5744 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:33.600629 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:33.601567 6552 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260430 19:59:33.853185 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:59:33.881165 6500 consensus_queue.cc:1048] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:33.959388 6556 raft_consensus.cc:493] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:33.959863 6556 raft_consensus.cc:515] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:33.961784 6556 leader_election.cc:290] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:33.964320 6064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb984dd833a443eac422b0821f4fa77" candidate_uuid: "cd13173a6c184acaa26b63bdbf8d6656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:33.965157 6064 raft_consensus.cc:2393] T fcb984dd833a443eac422b0821f4fa77 P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd13173a6c184acaa26b63bdbf8d6656 in current term 1: Already voted for candidate 787275a4c5c64a6aa81be209814867bf in this term.
I20260430 19:59:33.966181 6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb984dd833a443eac422b0821f4fa77" candidate_uuid: "cd13173a6c184acaa26b63bdbf8d6656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf" is_pre_election: true
I20260430 19:59:33.968633 6295 leader_election.cc:304] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 19:59:33.970029 6556 raft_consensus.cc:3060] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:33.977195 6556 raft_consensus.cc:2749] T fcb984dd833a443eac422b0821f4fa77 P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:34.108588 6558 consensus_queue.cc:1048] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:34.159271 6566 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f46e27ec79e44f329de14da21f60e60f)
I20260430 19:59:34.159776 6566 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:34.186100 6558 consensus_queue.cc:1048] T fcb984dd833a443eac422b0821f4fa77 P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:59:34.192495 6566 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:34.211776 5742 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:34.213577 5744 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:34.214049 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:34.214841 6566 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260430 19:59:34.290519 6570 consensus_queue.cc:1048] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260430 19:59:34.423075 6558 consensus_queue.cc:1048] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:59:34.899983 6544 debug-util.cc:398] Leaking SignalData structure 0x7b080002ecc0 after lost signal to thread 6473
W20260430 19:59:33.500512 6547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:35.112632 6473 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.613s user 0.585s sys 0.930s
W20260430 19:59:35.113500 6473 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.614s user 0.586s sys 0.930s
W20260430 19:59:35.115381 6549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:35.121013 6548 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1621 milliseconds
I20260430 19:59:35.121107 6473 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:59:35.122450 6473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:35.125154 6473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:35.126570 6473 hybrid_clock.cc:648] HybridClock initialized: now 1777579175126514 us; error 53 us; skew 500 ppm
I20260430 19:59:35.127538 6473 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:59:35.134405 6473 webserver.cc:492] Webserver started at http://127.3.190.60:37189/ using document root <none> and password file <none>
I20260430 19:59:35.135406 6473 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:35.135727 6473 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:35.143534 6473 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260430 19:59:35.149039 6584 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:35.150259 6473 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.003s
I20260430 19:59:35.150700 6473 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
uuid: "867d1f5512324df489edc5a61b79df54"
format_stamp: "Formatted at 2026-04-30 19:59:22 on dist-test-slave-6zcr"
I20260430 19:59:35.153388 6473 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:35.221230 6473 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:35.223217 6473 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:35.223775 6473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:35.291527 6625 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f46e27ec79e44f329de14da21f60e60f)
I20260430 19:59:35.292055 6625 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:35.293998 6625 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:35.299348 5742 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:35.300212 5744 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:35.300614 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:35.301331 6625 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260430 19:59:35.307448 6473 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.60:45507
I20260430 19:59:35.307526 6639 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.60:45507 every 8 connection(s)
I20260430 19:59:35.310117 6473 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-2/data/info.pb
I20260430 19:59:35.311549 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 6473
I20260430 19:59:35.324847 6640 sys_catalog.cc:263] Verifying existing consensus state
I20260430 19:59:35.332245 6640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Bootstrap starting.
I20260430 19:59:35.348265 6640 log.cc:826] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Log is configured to *not* fsync() on all Append() calls
I20260430 19:59:35.364377 6640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260430 19:59:35.365182 6640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Bootstrap complete.
I20260430 19:59:35.383852 6640 raft_consensus.cc:359] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:35.386483 6640 raft_consensus.cc:740] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867d1f5512324df489edc5a61b79df54, State: Initialized, Role: FOLLOWER
I20260430 19:59:35.387408 6640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:35.389851 6646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:35.390563 6646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:35.391269 6640 sys_catalog.cc:565] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: configured and running, proceeding with master startup.
I20260430 19:59:35.420120 6404 heartbeater.cc:344] Connected to a master server at 127.3.190.60:45507
I20260430 19:59:35.420644 6404 heartbeater.cc:499] Master 127.3.190.60:45507 was elected leader, sending a full tablet report...
I20260430 19:59:35.431334 6601 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" instance_seqno: 1777579170715407) as {username='slave'} at 127.3.190.3:48363; Asking this server to re-register.
I20260430 19:59:35.434055 6404 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:35.437587 6661 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:35.437853 6661 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: loading cluster ID for follower catalog manager: success
I20260430 19:59:35.437990 6601 ts_manager.cc:194] Registered new tserver with Master: cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:35.446404 6661 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: acquiring CA information for follower catalog manager: success
I20260430 19:59:35.453497 6661 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:59:35.468832 6259 heartbeater.cc:344] Connected to a master server at 127.3.190.60:45507
I20260430 19:59:35.474396 6601 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "787275a4c5c64a6aa81be209814867bf" instance_seqno: 1777579168472350) as {username='slave'} at 127.3.190.2:49849; Asking this server to re-register.
I20260430 19:59:35.475992 6259 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:35.483242 6601 ts_manager.cc:194] Registered new tserver with Master: 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:35.497926 6109 heartbeater.cc:344] Connected to a master server at 127.3.190.60:45507
I20260430 19:59:35.468559 6473 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 19:59:35.503304 6600 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "499a4336efe645d4926c94600c732b82" instance_seqno: 1777579166108385) as {username='slave'} at 127.3.190.1:40563; Asking this server to re-register.
I20260430 19:59:35.504267 6473 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:59:35.505065 6109 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:35.508549 6600 ts_manager.cc:194] Registered new tserver with Master: 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861)
I20260430 19:59:35.961987 6667 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:35.962550 6667 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
W20260430 19:59:35.975039 6585 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 19:59:35.975564 6667 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
W20260430 19:59:35.980230 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 19:59:35.982790 5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:35.983614 5767 raft_consensus.cc:2393] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 867d1f5512324df489edc5a61b79df54 in current term 5: Already voted for candidate f46e27ec79e44f329de14da21f60e60f in this term.
I20260430 19:59:35.984925 6585 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 867d1f5512324df489edc5a61b79df54; no voters: f2878444c3a64840b53f8d5f6fc6211d, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:35.985700 6667 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 4 FOLLOWER]: Advancing to term 5
I20260430 19:59:35.992763 6667 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260430 19:59:36.307981 6674 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f46e27ec79e44f329de14da21f60e60f)
I20260430 19:59:36.308488 6674 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:36.310418 6674 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:36.324329 5742 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260430 19:59:36.336580 5742 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 19:59:36.345643 6611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 6 candidate_status { last_received { term: 5 index: 30 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54" is_pre_election: true
I20260430 19:59:36.346519 6611 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2878444c3a64840b53f8d5f6fc6211d in term 5.
I20260430 19:59:36.347702 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters: f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:36.348512 6674 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260430 19:59:36.348877 6674 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Starting leader election (detected failure of leader f46e27ec79e44f329de14da21f60e60f)
I20260430 19:59:36.349231 6674 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 5 FOLLOWER]: Advancing to term 6
I20260430 19:59:36.357252 6674 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:36.359354 6674 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 election: Requested vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
I20260430 19:59:36.360253 6611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2878444c3a64840b53f8d5f6fc6211d" candidate_term: 6 candidate_status { last_received { term: 5 index: 30 } } ignore_live_leader: false dest_uuid: "867d1f5512324df489edc5a61b79df54"
I20260430 19:59:36.360764 6611 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 5 FOLLOWER]: Advancing to term 6
W20260430 19:59:36.364259 5742 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 19:59:36.367471 6611 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2878444c3a64840b53f8d5f6fc6211d in term 6.
I20260430 19:59:36.370962 5744 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters: f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:36.372126 6674 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 6 FOLLOWER]: Leader election won for term 6
I20260430 19:59:36.375257 6674 raft_consensus.cc:697] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 6 LEADER]: Becoming Leader. State: Replica: f2878444c3a64840b53f8d5f6fc6211d, State: Running, Role: LEADER
I20260430 19:59:36.376052 6674 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 5.30, Last appended by leader: 30, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:36.379729 6679 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2878444c3a64840b53f8d5f6fc6211d. Latest consensus state: current_term: 6 leader_uuid: "f2878444c3a64840b53f8d5f6fc6211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:36.380472 6679 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:36.382696 6681 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:59:36.391278 6681 catalog_manager.cc:679] Loaded metadata for table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]
I20260430 19:59:36.391568 6682 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:36.392009 6682 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.393035 6681 catalog_manager.cc:679] Loaded metadata for table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]
I20260430 19:59:36.393819 6682 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:36.394732 6681 catalog_manager.cc:679] Loaded metadata for table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]
W20260430 19:59:36.395534 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 19:59:36.396315 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:36.396701 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 19:59:36.397377 6682 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:36.405990 6681 tablet_loader.cc:96] loaded metadata for tablet 69deeb4a53f04b1b87be5d406890d10b (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d])
I20260430 19:59:36.407791 6681 tablet_loader.cc:96] loaded metadata for tablet 87720d2b47d6419bb1420c6f0187d74b (table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d])
I20260430 19:59:36.409152 6681 tablet_loader.cc:96] loaded metadata for tablet b26883f75c5b4b3480bbc43cfd03e1ee (table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb])
I20260430 19:59:36.412904 6681 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:59:36.416213 6681 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:36.416540 6681 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:59:36.420954 6681 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:59:36.424124 6681 catalog_manager.cc:6055] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Loaded TSK: 0
I20260430 19:59:36.424888 6681 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:59:36.428443 6681 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260430 19:59:36.431425 6681 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260430 19:59:36.455425 6611 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Refusing update from remote peer f2878444c3a64840b53f8d5f6fc6211d: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 32. (index mismatch)
I20260430 19:59:36.456923 6674 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 5, Time since last communication: 0.000s
W20260430 19:59:36.457705 5742 consensus_peers.cc:597] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d -> Peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Couldn't send request to peer f46e27ec79e44f329de14da21f60e60f. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:59:36.494542 5757 master_service.cc:954] ReplaceTablet: received request to replace tablet 69deeb4a53f04b1b87be5d406890d10b from {username='slave'} at 127.0.0.1:53392
I20260430 19:59:36.497469 5756 master_service.cc:954] ReplaceTablet: received request to replace tablet 69deeb4a53f04b1b87be5d406890d10b from {username='slave'} at 127.0.0.1:53392
I20260430 19:59:36.499011 5753 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53392:
name: "default.Table_955554de498d41cebb30bea37e154c53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:36.500052 5754 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53392:
name: "default.Table_c9b777e92e10430b8f8d1920f4ffc8bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:36.499085 5755 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53392:
name: "default.Table_8868e9393dc8483eb17d875dc93821ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260430 19:59:36.512573 5915 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1777579171, eventType=CREATE_TABLE, dbName=default, tableName=table_a4a99cd76d004be8a0e4b66d9e74fa77, message={"server":"","servicePrincipal":"","db":"default","table":"table_a4a99cd76d004be8a0e4b66d9e74fa77","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a4a99cd76d004be8a0e4b66d9e74fa77\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777579171},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/warehouse/table_a4a99cd76d004be8a0e4b66d9e74fa77\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a4a99cd76d004be8a0e4b66d9e74fa77\",\"kudu.table_id\":\"f29979fa635848d98e8466f0a3576a6d\",\"transient_lastDdlTime\":\"1777579171\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4963134f6a694793a4115ae1c2aa919c\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777579171,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777579171, eventType=CREATE_TABLE, dbName=default, tableName=table_1f40269e3dc346979445899c9c93ac54, message={"server":"","servicePrincipal":"","db":"default","table":"table_1f40269e3dc346979445899c9c93ac54","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_1f40269e3dc346979445899c9c93ac54\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777579171},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/warehouse/table_1f40269e3dc346979445899c9c93ac54\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_1f40269e3dc346979445899c9c93ac54\",\"kudu.table_id\":\"7a6137ba6a5046ec945d466836f35e0a\",\"transient_lastDdlTime\":\"1777579171\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4963134f6a694793a4115ae1c2aa919c\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777579171,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260430 19:59:36.512905 5751 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53392:
table { table_name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7" } modify_external_catalogs: true
I20260430 19:59:36.516690 5751 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53392:
table { table_name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7" } modify_external_catalogs: true
I20260430 19:59:36.516566 5752 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53392:
table { table_name: "default.Table_5ef46544dc424dea9756db410f1a18c1" } new_table_name: "default.Table_83c2669f3f9548348584e0ecd6d59c9d" modify_external_catalogs: true
I20260430 19:59:36.518626 5750 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53392:
table { table_name: "default.Table_a4a99cd76d004be8a0e4b66d9e74fa77" } new_table_name: "default.Table_8bf37998afb24f6db71c7e78a44337dc" modify_external_catalogs: true
I20260430 19:59:36.503593 5758 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53392:
name: "default.Table_b2b7ae30e3404fe8be26c7cc54e7d102"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:36.550238 6684 mvcc.cc:204] Tried to move back new op lower bound from 7280964306746433536 to 7280964306442522624. Current Snapshot: MvccSnapshot[applied={T|T < 7280964306746433536}]
I20260430 19:59:36.556310 6667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2878444c3a64840b53f8d5f6fc6211d. Latest consensus state: current_term: 6 leader_uuid: "f2878444c3a64840b53f8d5f6fc6211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:36.557025 6667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:36.554499 6688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "f2878444c3a64840b53f8d5f6fc6211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:36.559721 6688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:36.568537 6674 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "f2878444c3a64840b53f8d5f6fc6211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:36.570408 6674 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:36.570973 6667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "f2878444c3a64840b53f8d5f6fc6211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:36.571640 6667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f2dbae56950 google::LogMessage::Flush() at ??:0
@ 0x7f2dbae566bf google::LogMessage::~LogMessage() at ??:0
I20260430 19:59:36.625053 6339 tablet_service.cc:1511] Processing CreateTablet for tablet 87720d2b47d6419bb1420c6f0187d74b (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:36.626186 6339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87720d2b47d6419bb1420c6f0187d74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:36.629817 5757 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Sending DeleteTablet for 3 replicas of tablet 69deeb4a53f04b1b87be5d406890d10b
I20260430 19:59:36.631601 5757 catalog_manager.cc:6741] ReplaceTablet: tablet 69deeb4a53f04b1b87be5d406890d10b deleted and replaced by tablet d00366d491b247468adcee8fbf41f0a1
I20260430 19:59:36.636737 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet 69deeb4a53f04b1b87be5d406890d10b with delete_type TABLET_DATA_DELETED (replaced by tablet d00366d491b247468adcee8fbf41f0a1) from {username='slave'} at 127.0.0.1:50872
I20260430 19:59:36.651010 6044 tablet_service.cc:1511] Processing CreateTablet for tablet 87720d2b47d6419bb1420c6f0187d74b (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:36.652395 6044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87720d2b47d6419bb1420c6f0187d74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:36.660378 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet 69deeb4a53f04b1b87be5d406890d10b with delete_type TABLET_DATA_DELETED (replaced by tablet d00366d491b247468adcee8fbf41f0a1) from {username='slave'} at 127.0.0.1:33962
I20260430 19:59:36.669132 6704 tablet_replica.cc:333] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:36.669883 6704 raft_consensus.cc:2243] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Raft consensus shutting down.
@ 0x7f2dc5f0d2bc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260430 19:59:36.685144 6702 tablet_replica.cc:333] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
I20260430 19:59:36.685907 6702 raft_consensus.cc:2243] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:36.686435 6702 raft_consensus.cc:2272] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:36.670472 6704 raft_consensus.cc:2272] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:36.688496 6702 ts_tablet_manager.cc:1916] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:36.688357 6685 mvcc.cc:204] Tried to move back new op lower bound from 7280964306746433536 to 7280964284676235264. Current Snapshot: MvccSnapshot[applied={T|T < 7280964306746433536 or (T in {7280964306746433536})}]
I20260430 19:59:36.689088 6704 ts_tablet_manager.cc:1916] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:36.697049 6043 tablet_service.cc:1511] Processing CreateTablet for tablet d00366d491b247468adcee8fbf41f0a1 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:36.698205 6043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d00366d491b247468adcee8fbf41f0a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:36.704475 6338 tablet_service.cc:1511] Processing CreateTablet for tablet d00366d491b247468adcee8fbf41f0a1 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:36.710914 6338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d00366d491b247468adcee8fbf41f0a1. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f2dc5f0ca7c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260430 19:59:36.737522 6706 tablet_bootstrap.cc:492] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:36.740698 6192 tablet_service.cc:1511] Processing CreateTablet for tablet d00366d491b247468adcee8fbf41f0a1 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:36.743110 6193 tablet_service.cc:1558] Processing DeleteTablet for tablet 69deeb4a53f04b1b87be5d406890d10b with delete_type TABLET_DATA_DELETED (replaced by tablet d00366d491b247468adcee8fbf41f0a1) from {username='slave'} at 127.0.0.1:44480
I20260430 19:59:36.745396 6194 tablet_service.cc:1511] Processing CreateTablet for tablet 87720d2b47d6419bb1420c6f0187d74b (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:36.746552 6192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d00366d491b247468adcee8fbf41f0a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:36.750893 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87720d2b47d6419bb1420c6f0187d74b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:59:36.754549 6149 consensus_peers.cc:597] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:59:36.760689 6708 tablet_bootstrap.cc:492] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
@ 0x7f2dc5f10af2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260430 19:59:36.789373 6708 tablet_bootstrap.cc:654] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:36.792735 6710 tablet_replica.cc:333] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:36.793853 6710 raft_consensus.cc:2243] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:59:36.795258 6710 raft_consensus.cc:2272] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:36.798297 6710 ts_tablet_manager.cc:1916] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:36.805022 6704 ts_tablet_manager.cc:1929] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:36.805451 6704 log.cc:1199] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/69deeb4a53f04b1b87be5d406890d10b
I20260430 19:59:36.806321 6704 ts_tablet_manager.cc:1950] T 69deeb4a53f04b1b87be5d406890d10b P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:36.808077 6714 tablet_bootstrap.cc:492] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:36.808960 5744 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet 69deeb4a53f04b1b87be5d406890d10b (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:36.810483 6706 tablet_bootstrap.cc:654] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:36.814222 6708 tablet_bootstrap.cc:492] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:36.814755 6708 ts_tablet_manager.cc:1403] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.004s
I20260430 19:59:36.817407 6710 ts_tablet_manager.cc:1929] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:36.817854 6710 log.cc:1199] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/69deeb4a53f04b1b87be5d406890d10b
I20260430 19:59:36.817865 6702 ts_tablet_manager.cc:1929] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:36.817560 6708 raft_consensus.cc:359] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.818681 6708 raft_consensus.cc:385] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:36.818751 6710 ts_tablet_manager.cc:1950] T 69deeb4a53f04b1b87be5d406890d10b P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:36.818974 6708 raft_consensus.cc:740] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:36.821593 5745 catalog_manager.cc:5002] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): tablet 69deeb4a53f04b1b87be5d406890d10b (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:36.818228 6702 log.cc:1199] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/69deeb4a53f04b1b87be5d406890d10b
I20260430 19:59:36.823333 6702 ts_tablet_manager.cc:1950] T 69deeb4a53f04b1b87be5d406890d10b P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:36.822881 6708 consensus_queue.cc:260] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.826934 6708 ts_tablet_manager.cc:1434] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.012s user 0.001s sys 0.006s
I20260430 19:59:36.827760 6708 tablet_bootstrap.cc:492] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:36.828688 5744 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet 69deeb4a53f04b1b87be5d406890d10b (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:36.829739 6714 tablet_bootstrap.cc:654] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
@ 0x7f2dc5f10aaa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260430 19:59:36.833649 6708 tablet_bootstrap.cc:654] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:36.847289 5757 master_service.cc:954] ReplaceTablet: received request to replace tablet b26883f75c5b4b3480bbc43cfd03e1ee from {username='slave'} at 127.0.0.1:53392
I20260430 19:59:36.853042 6405 heartbeater.cc:499] Master 127.3.190.61:35031 was elected leader, sending a full tablet report...
W20260430 19:59:36.859647 5749 catalog_manager.cc:5504] Ignoring report from unknown tablet d28c8f73e1bd4acd8e30805e34662440
W20260430 19:59:36.859887 5749 catalog_manager.cc:5504] Ignoring report from unknown tablet fcb984dd833a443eac422b0821f4fa77
W20260430 19:59:36.860086 5749 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:36.859477 6708 tablet_bootstrap.cc:492] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
@ 0x7f2dc5f10a3a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260430 19:59:36.860759 6708 ts_tablet_manager.cc:1403] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.008s
I20260430 19:59:36.863538 6708 raft_consensus.cc:359] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.863958 6706 tablet_bootstrap.cc:492] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:36.864213 5756 master_service.cc:954] ReplaceTablet: received request to replace tablet b26883f75c5b4b3480bbc43cfd03e1ee from {username='slave'} at 127.0.0.1:53392
I20260430 19:59:36.864305 6708 raft_consensus.cc:385] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:36.864478 6706 ts_tablet_manager.cc:1403] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.127s user 0.000s sys 0.011s
I20260430 19:59:36.864580 6708 raft_consensus.cc:740] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:36.866526 6682 raft_consensus.cc:493] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:36.866972 6682 raft_consensus.cc:515] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.867419 6706 raft_consensus.cc:359] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.868222 6706 raft_consensus.cc:385] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:36.868506 6706 raft_consensus.cc:740] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:36.868847 6682 leader_election.cc:290] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:36.869170 6706 consensus_queue.cc:260] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.871001 6064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87720d2b47d6419bb1420c6f0187d74b" candidate_uuid: "cd13173a6c184acaa26b63bdbf8d6656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:36.871608 6064 raft_consensus.cc:2468] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd13173a6c184acaa26b63bdbf8d6656 in term 0.
I20260430 19:59:36.872784 6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87720d2b47d6419bb1420c6f0187d74b" candidate_uuid: "cd13173a6c184acaa26b63bdbf8d6656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf" is_pre_election: true
I20260430 19:59:36.874372 6294 leader_election.cc:304] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656; no voters:
W20260430 19:59:36.875643 6295 leader_election.cc:343] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:59:36.877535 6682 raft_consensus.cc:2804] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:36.877864 6682 raft_consensus.cc:493] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:36.878157 6682 raft_consensus.cc:3060] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Advancing to term 1
@ 0x7f2dc5f10a02 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260430 19:59:36.882931 6708 consensus_queue.cc:260] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.890470 6708 ts_tablet_manager.cc:1434] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.029s user 0.006s sys 0.004s
I20260430 19:59:36.900040 5757 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Sending DeleteTablet for 3 replicas of tablet b26883f75c5b4b3480bbc43cfd03e1ee
I20260430 19:59:36.901774 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet b26883f75c5b4b3480bbc43cfd03e1ee with delete_type TABLET_DATA_DELETED (replaced by tablet f331ae400bff46f49524f28b87e006ad) from {username='slave'} at 127.0.0.1:33962
I20260430 19:59:36.902431 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet b26883f75c5b4b3480bbc43cfd03e1ee with delete_type TABLET_DATA_DELETED (replaced by tablet f331ae400bff46f49524f28b87e006ad) from {username='slave'} at 127.0.0.1:44480
I20260430 19:59:36.902551 6704 tablet_replica.cc:333] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:36.903256 6704 raft_consensus.cc:2243] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:36.903715 6704 raft_consensus.cc:2272] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260430 19:59:36.903975 5745 catalog_manager.cc:4969] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): delete failed for tablet b26883f75c5b4b3480bbc43cfd03e1ee because the tablet was not found. No further retry: Not found: Tablet not found: b26883f75c5b4b3480bbc43cfd03e1ee
I20260430 19:59:36.905629 6704 ts_tablet_manager.cc:1916] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:36.906042 6682 raft_consensus.cc:515] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.908737 6064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87720d2b47d6419bb1420c6f0187d74b" candidate_uuid: "cd13173a6c184acaa26b63bdbf8d6656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82"
I20260430 19:59:36.908959 6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87720d2b47d6419bb1420c6f0187d74b" candidate_uuid: "cd13173a6c184acaa26b63bdbf8d6656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf"
I20260430 19:59:36.909308 6064 raft_consensus.cc:3060] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
W20260430 19:59:36.910248 6295 leader_election.cc:343] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Illegal state: must be running to vote when last-logged opid is not known
I20260430 19:59:36.913930 6260 heartbeater.cc:499] Master 127.3.190.61:35031 was elected leader, sending a full tablet report...
I20260430 19:59:36.916226 6064 raft_consensus.cc:2468] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd13173a6c184acaa26b63bdbf8d6656 in term 1.
I20260430 19:59:36.917398 6294 leader_election.cc:304] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656; no voters: 787275a4c5c64a6aa81be209814867bf
I20260430 19:59:36.918182 6721 raft_consensus.cc:2804] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Leader election won for term 1
@ 0x7f2dc5f0fcfe std::__1::__function::__func<>::operator()() at ??:0
@ 0x5b3bc5 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5b39fa std::__1::function<>::operator()() at ??:?
@ 0x7f2dbd07cfbb kudu::Thread::SuperviseThread() at ??:0
@ 0x479510 __tsan_thread_start_func at ??:?
I20260430 19:59:36.948333 5757 catalog_manager.cc:6741] ReplaceTablet: tablet b26883f75c5b4b3480bbc43cfd03e1ee deleted and replaced by tablet f331ae400bff46f49524f28b87e006ad
I20260430 19:59:36.950380 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet b26883f75c5b4b3480bbc43cfd03e1ee with delete_type TABLET_DATA_DELETED (replaced by tablet f331ae400bff46f49524f28b87e006ad) from {username='slave'} at 127.0.0.1:50872
W20260430 19:59:36.953192 5744 catalog_manager.cc:4969] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): delete failed for tablet b26883f75c5b4b3480bbc43cfd03e1ee because the tablet was not found. No further retry: Not found: Tablet not found: b26883f75c5b4b3480bbc43cfd03e1ee
I20260430 19:59:36.952746 6721 raft_consensus.cc:697] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 LEADER]: Becoming Leader. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Running, Role: LEADER
I20260430 19:59:36.954905 6721 consensus_queue.cc:237] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
W20260430 19:59:36.955946 5749 catalog_manager.cc:5504] Ignoring report from unknown tablet fcb984dd833a443eac422b0821f4fa77
W20260430 19:59:36.957558 5749 catalog_manager.cc:5504] Ignoring report from unknown tablet fb30716bb9174641ba41885f792e6364
W20260430 19:59:36.957758 5749 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:36.959354 6704 ts_tablet_manager.cc:1929] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260430 19:59:36.959729 6704 log.cc:1199] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/b26883f75c5b4b3480bbc43cfd03e1ee
I20260430 19:59:36.960356 6704 ts_tablet_manager.cc:1950] T b26883f75c5b4b3480bbc43cfd03e1ee P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:36.961738 6714 tablet_bootstrap.cc:492] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:36.962185 6714 ts_tablet_manager.cc:1403] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.154s user 0.009s sys 0.009s
@ 0x7f2dbe76f6db start_thread at ??:0
I20260430 19:59:36.965225 6714 raft_consensus.cc:359] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:36.965939 6714 raft_consensus.cc:385] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:36.966233 6714 raft_consensus.cc:740] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:36.968605 6043 tablet_service.cc:1511] Processing CreateTablet for tablet f331ae400bff46f49524f28b87e006ad (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:36.969694 6043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f331ae400bff46f49524f28b87e006ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:36.982687 5744 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet b26883f75c5b4b3480bbc43cfd03e1ee (table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]) successfully deleted
I20260430 19:59:36.991324 5757 catalog_manager.cc:5671] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3). New cstate: current_term: 1 leader_uuid: "cd13173a6c184acaa26b63bdbf8d6656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } health_report { overall_health: HEALTHY } } }
@ 0x7f2db9afb61f clone at ??:0
I20260430 19:59:37.000483 6682 leader_election.cc:290] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 election: Requested vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:37.008687 6110 heartbeater.cc:499] Master 127.3.190.61:35031 was elected leader, sending a full tablet report...
I20260430 19:59:37.008773 6714 consensus_queue.cc:260] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.009783 6194 tablet_service.cc:1511] Processing CreateTablet for tablet f331ae400bff46f49524f28b87e006ad (DEFAULT_TABLE table=default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:37.010852 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f331ae400bff46f49524f28b87e006ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:37.017832 6706 ts_tablet_manager.cc:1434] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.153s user 0.013s sys 0.023s
I20260430 19:59:37.021230 6706 tablet_bootstrap.cc:492] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:37.023965 6714 ts_tablet_manager.cc:1434] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.061s user 0.008s sys 0.000s
I20260430 19:59:37.031133 6714 tablet_bootstrap.cc:492] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:37.034335 6706 tablet_bootstrap.cc:654] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:37.037505 6714 tablet_bootstrap.cc:654] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:37.057266 6706 tablet_bootstrap.cc:492] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:37.057705 6706 ts_tablet_manager.cc:1403] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.002s
I20260430 19:59:37.060650 6706 raft_consensus.cc:359] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.061352 6706 raft_consensus.cc:385] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:37.061609 6706 raft_consensus.cc:740] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:37.062209 6706 consensus_queue.cc:260] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.065133 6706 ts_tablet_manager.cc:1434] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260430 19:59:37.066063 6706 tablet_bootstrap.cc:492] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:37.071234 6714 tablet_bootstrap.cc:492] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:37.071660 6714 ts_tablet_manager.cc:1403] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.041s user 0.016s sys 0.016s
I20260430 19:59:37.072863 6706 tablet_bootstrap.cc:654] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:37.075004 6714 raft_consensus.cc:359] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.075649 6714 raft_consensus.cc:385] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:37.075909 6714 raft_consensus.cc:740] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:37.076619 6714 consensus_queue.cc:260] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.082813 6714 ts_tablet_manager.cc:1434] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.011s user 0.006s sys 0.001s
I20260430 19:59:37.088760 6714 tablet_bootstrap.cc:492] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:37.096872 6714 tablet_bootstrap.cc:654] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:37.107831 6706 tablet_bootstrap.cc:492] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:37.108307 6706 ts_tablet_manager.cc:1403] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.042s user 0.023s sys 0.002s
I20260430 19:59:37.109614 6714 tablet_bootstrap.cc:492] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:37.110031 6714 ts_tablet_manager.cc:1403] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.007s
I20260430 19:59:37.110911 6706 raft_consensus.cc:359] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.111589 6706 raft_consensus.cc:385] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:37.111856 6706 raft_consensus.cc:740] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:37.112694 6714 raft_consensus.cc:359] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.113344 6714 raft_consensus.cc:385] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:37.113607 6714 raft_consensus.cc:740] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:37.113298 6706 consensus_queue.cc:260] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.117082 6706 ts_tablet_manager.cc:1434] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260430 19:59:37.118294 6714 consensus_queue.cc:260] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.124325 6714 ts_tablet_manager.cc:1434] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260430 19:59:37.136023 6720 raft_consensus.cc:493] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:37.136535 6720 raft_consensus.cc:515] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.138619 6720 leader_election.cc:290] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:37.140211 6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00366d491b247468adcee8fbf41f0a1" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf" is_pre_election: true
I20260430 19:59:37.140496 6358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00366d491b247468adcee8fbf41f0a1" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
I20260430 19:59:37.140952 6214 raft_consensus.cc:2468] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 499a4336efe645d4926c94600c732b82 in term 0.
I20260430 19:59:37.141163 6358 raft_consensus.cc:2468] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 499a4336efe645d4926c94600c732b82 in term 0.
I20260430 19:59:37.142570 6000 leader_election.cc:304] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:37.142745 6673 raft_consensus.cc:493] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:37.143227 6673 raft_consensus.cc:515] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.143569 6720 raft_consensus.cc:2804] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:37.143939 6720 raft_consensus.cc:493] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:37.144290 6720 raft_consensus.cc:3060] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:37.145258 6673 leader_election.cc:290] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 19:59:37.150990 6149 leader_election.cc:343] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: f331ae400bff46f49524f28b87e006ad
I20260430 19:59:37.154712 6063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f331ae400bff46f49524f28b87e006ad" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:37.155345 6063 raft_consensus.cc:2468] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:37.157636 6149 leader_election.cc:304] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters: cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:37.158692 6673 raft_consensus.cc:2804] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:37.159004 6673 raft_consensus.cc:493] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:37.159294 6673 raft_consensus.cc:3060] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:37.167315 6709 raft_consensus.cc:493] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:37.167763 6709 raft_consensus.cc:515] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.169584 6709 leader_election.cc:290] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:37.171701 6358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87720d2b47d6419bb1420c6f0187d74b" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
I20260430 19:59:37.174321 6063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87720d2b47d6419bb1420c6f0187d74b" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:37.175058 6063 raft_consensus.cc:2393] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787275a4c5c64a6aa81be209814867bf in current term 1: Already voted for candidate cd13173a6c184acaa26b63bdbf8d6656 in this term.
I20260430 19:59:37.176245 6149 leader_election.cc:304] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:37.177384 6709 raft_consensus.cc:3060] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:37.166145 6673 raft_consensus.cc:515] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.185261 6709 raft_consensus.cc:2749] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:37.188249 6720 raft_consensus.cc:515] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
W20260430 19:59:37.193286 6149 leader_election.cc:343] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: f331ae400bff46f49524f28b87e006ad
I20260430 19:59:37.193660 6673 leader_election.cc:290] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Requested vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:37.190488 6063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f331ae400bff46f49524f28b87e006ad" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82"
I20260430 19:59:37.194954 6063 raft_consensus.cc:3060] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:37.202626 6063 raft_consensus.cc:2468] T f331ae400bff46f49524f28b87e006ad P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:37.210131 6149 leader_election.cc:304] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters: cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:37.211397 6673 raft_consensus.cc:2804] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:37.211908 6673 raft_consensus.cc:697] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [term 1 LEADER]: Becoming Leader. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Running, Role: LEADER
I20260430 19:59:37.218729 6214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00366d491b247468adcee8fbf41f0a1" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787275a4c5c64a6aa81be209814867bf"
I20260430 19:59:37.219264 6214 raft_consensus.cc:3060] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:37.225832 6214 raft_consensus.cc:2468] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 499a4336efe645d4926c94600c732b82 in term 1.
I20260430 19:59:37.227061 6000 leader_election.cc:304] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:37.216383 6673 consensus_queue.cc:237] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.228617 6737 raft_consensus.cc:2804] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:37.231544 6358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d00366d491b247468adcee8fbf41f0a1" candidate_uuid: "499a4336efe645d4926c94600c732b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656"
I20260430 19:59:37.232085 6358 raft_consensus.cc:3060] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:37.237191 6737 raft_consensus.cc:697] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 1 LEADER]: Becoming Leader. State: Replica: 499a4336efe645d4926c94600c732b82, State: Running, Role: LEADER
I20260430 19:59:37.238191 6737 consensus_queue.cc:237] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:37.241091 6358 raft_consensus.cc:2468] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 499a4336efe645d4926c94600c732b82 in term 1.
I20260430 19:59:37.242434 6720 leader_election.cc:290] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [CANDIDATE]: Term 1 election: Requested vote from peers 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:37.268491 6673 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:37.269075 6673 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
W20260430 19:59:37.272464 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:37.272902 6673 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 19:59:37.273141 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:37.273537 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:37.274251 6673 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 19:59:37.301373 6148 connection.cc:570] server connection from 127.0.0.1:44480 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:37.302136 5997 connection.cc:570] server connection from 127.0.0.1:33962 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:37.302184 6147 connection.cc:570] client connection to 127.3.190.61:35031 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:37.302235 6414 connection.cc:570] client connection to 127.3.190.61:35031 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:37.303553 5997 connection.cc:570] client connection to 127.3.190.61:35031 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:37.304157 6414 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.190.61:35031)
W20260430 19:59:37.304817 6414 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.190.61:35031)
W20260430 19:59:37.305430 6414 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.190.61:35031)
W20260430 19:59:37.306336 6414 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.190.61:35031)
W20260430 19:59:37.316439 6727 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:59:37.307910 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.61:35031 (local address 127.3.190.1:60163)
0430 19:59:37.311373 (+ 3463us) negotiation.cc:107] Waiting for socket to connect
0430 19:59:37.311418 (+ 45us) client_negotiation.cc:175] Beginning negotiation
0430 19:59:37.311668 (+ 250us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0430 19:59:37.315256 (+ 3588us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2843}
W20260430 19:59:37.316761 6711 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:59:37.304217 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.61:35031 (local address 127.3.190.2:41941)
0430 19:59:37.308414 (+ 4197us) negotiation.cc:107] Waiting for socket to connect
0430 19:59:37.308474 (+ 60us) client_negotiation.cc:175] Beginning negotiation
0430 19:59:37.308811 (+ 337us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0430 19:59:37.315598 (+ 6787us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":3609}
I20260430 19:59:37.449870 6743 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2878444c3a64840b53f8d5f6fc6211d)
I20260430 19:59:37.450356 6743 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:37.452105 6743 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
W20260430 19:59:37.460292 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:37.460770 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031): Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:37.461082 6585 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 867d1f5512324df489edc5a61b79df54; no voters: f2878444c3a64840b53f8d5f6fc6211d, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:37.461653 6743 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260430 19:59:37.965839 6751 consensus_queue.cc:1048] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:37.982913 6755 consensus_queue.cc:1048] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260430 19:59:38.146051 6149 consensus_peers.cc:597] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf -> Peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Couldn't send request to peer cd13173a6c184acaa26b63bdbf8d6656. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f331ae400bff46f49524f28b87e006ad. This is attempt 1: this message will repeat every 5th retry.
I20260430 19:59:38.156303 6748 consensus_queue.cc:1048] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:38.168010 6762 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2878444c3a64840b53f8d5f6fc6211d)
I20260430 19:59:38.168574 6762 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:38.177381 6762 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
W20260430 19:59:38.181304 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:38.182053 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031): Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:38.182458 6585 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 867d1f5512324df489edc5a61b79df54; no voters: f2878444c3a64840b53f8d5f6fc6211d, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:38.183192 6762 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260430 19:59:38.264763 6752 consensus_queue.cc:1048] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:38.283756 6766 consensus_queue.cc:1048] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:38.319562 3832 external_mini_cluster.cc:1366] Running /tmp/dist-test-task07fIBA/build/tsan/bin/kudu
/tmp/dist-test-task07fIBA/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.61:35031
--webserver_interface=127.3.190.61
--webserver_port=45235
--builtin_ntp_servers=127.3.190.20:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--hive_metastore_uris=thrift://127.0.0.1:32957
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260430 19:59:38.647429 6769 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260430 19:59:38.648016 6769 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260430 19:59:38.648553 6769 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260430 19:59:38.684942 6769 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260430 19:59:38.685207 6769 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260430 19:59:38.685420 6769 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260430 19:59:38.685632 6769 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260430 19:59:38.685849 6769 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260430 19:59:38.686070 6769 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260430 19:59:38.728060 6769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.190.20:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:32957
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.190.61:35031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.190.61
--webserver_port=45235
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 8537ed5bb5e0e6ac1e481ff961e6b9d08cce7671
build type FASTDEBUG
built by None at 30 Apr 2026 19:43:24 UTC on e7f111948823
build id 11683
TSAN enabled
I20260430 19:59:38.729415 6769 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260430 19:59:38.731156 6769 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260430 19:59:38.743243 6775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260430 19:59:39.256163 6797 ts_tablet_manager.cc:933] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: Initiating tablet copy from peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:39.263748 6798 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2878444c3a64840b53f8d5f6fc6211d)
I20260430 19:59:39.265494 6798 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:39.269258 6797 tablet_copy_client.cc:323] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: tablet copy: Beginning tablet copy session from remote peer at address 127.3.190.2:35955
I20260430 19:59:39.271951 6234 tablet_copy_service.cc:140] P 787275a4c5c64a6aa81be209814867bf: Received BeginTabletCopySession request for tablet f331ae400bff46f49524f28b87e006ad from peer cd13173a6c184acaa26b63bdbf8d6656 ({username='slave'} at 127.3.190.3:35845)
I20260430 19:59:39.272567 6234 tablet_copy_service.cc:161] P 787275a4c5c64a6aa81be209814867bf: Beginning new tablet copy session on tablet f331ae400bff46f49524f28b87e006ad from peer cd13173a6c184acaa26b63bdbf8d6656 at {username='slave'} at 127.3.190.3:35845: session id = cd13173a6c184acaa26b63bdbf8d6656-f331ae400bff46f49524f28b87e006ad
I20260430 19:59:39.279065 6798 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
I20260430 19:59:39.281569 6234 tablet_copy_source_session.cc:215] T f331ae400bff46f49524f28b87e006ad P 787275a4c5c64a6aa81be209814867bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260430 19:59:39.287371 6797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f331ae400bff46f49524f28b87e006ad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260430 19:59:39.300464 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:39.304777 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031): Network error: Client connection negotiation failed: client connection to 127.3.190.61:35031: connect: Connection refused (error 111)
I20260430 19:59:39.305315 6585 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 867d1f5512324df489edc5a61b79df54; no voters: f2878444c3a64840b53f8d5f6fc6211d, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:39.306092 6798 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260430 19:59:39.318698 6797 tablet_copy_client.cc:806] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: tablet copy: Starting download of 0 data blocks...
I20260430 19:59:39.319478 6797 tablet_copy_client.cc:670] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: tablet copy: Starting download of 1 WAL segments...
I20260430 19:59:39.330068 6797 tablet_copy_client.cc:538] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260430 19:59:39.356822 6797 tablet_bootstrap.cc:492] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:39.378918 6797 tablet_bootstrap.cc:492] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260430 19:59:39.379998 6797 tablet_bootstrap.cc:492] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap complete.
I20260430 19:59:39.380759 6797 ts_tablet_manager.cc:1403] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.024s user 0.020s sys 0.003s
I20260430 19:59:39.384526 6797 raft_consensus.cc:359] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:39.385514 6797 raft_consensus.cc:740] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:39.386361 6797 consensus_queue.cc:260] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:39.389810 6797 ts_tablet_manager.cc:1434] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260430 19:59:39.392742 6234 tablet_copy_service.cc:342] P 787275a4c5c64a6aa81be209814867bf: Request end of tablet copy session cd13173a6c184acaa26b63bdbf8d6656-f331ae400bff46f49524f28b87e006ad received from {username='slave'} at 127.3.190.3:35845
I20260430 19:59:39.393419 6234 tablet_copy_service.cc:434] P 787275a4c5c64a6aa81be209814867bf: ending tablet copy session cd13173a6c184acaa26b63bdbf8d6656-f331ae400bff46f49524f28b87e006ad on tablet f331ae400bff46f49524f28b87e006ad with peer cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:40.002228 6358 raft_consensus.cc:1217] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
W20260430 19:59:38.744176 6776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:40.055779 6778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260430 19:59:40.061369 6777 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1309 milliseconds
I20260430 19:59:40.061539 6769 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260430 19:59:40.062882 6769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260430 19:59:40.065676 6769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260430 19:59:40.067147 6769 hybrid_clock.cc:648] HybridClock initialized: now 1777579180067078 us; error 104 us; skew 500 ppm
I20260430 19:59:40.068123 6769 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260430 19:59:40.074477 6769 webserver.cc:492] Webserver started at http://127.3.190.61:45235/ using document root <none> and password file <none>
I20260430 19:59:40.075460 6769 fs_manager.cc:362] Metadata directory not provided
I20260430 19:59:40.075711 6769 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260430 19:59:40.084750 6769 fs_manager.cc:714] Time spent opening directory manager: real 0.006s user 0.001s sys 0.004s
I20260430 19:59:40.089576 6817 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260430 19:59:40.090896 6769 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260430 19:59:40.091282 6769 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data,/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
uuid: "f2878444c3a64840b53f8d5f6fc6211d"
format_stamp: "Formatted at 2026-04-30 19:59:20 on dist-test-slave-6zcr"
I20260430 19:59:40.093240 6769 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260430 19:59:40.152421 6769 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260430 19:59:40.153970 6769 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260430 19:59:40.154454 6769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260430 19:59:40.233887 6769 rpc_server.cc:307] RPC server started. Bound to: 127.3.190.61:35031
I20260430 19:59:40.233975 6868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.190.61:35031 every 8 connection(s)
I20260430 19:59:40.236950 6769 server_base.cc:1193] Dumped server information to /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/master-1/data/info.pb
I20260430 19:59:40.240216 3832 external_mini_cluster.cc:1428] Started /tmp/dist-test-task07fIBA/build/tsan/bin/kudu as pid 6769
I20260430 19:59:40.249701 6869 sys_catalog.cc:263] Verifying existing consensus state
I20260430 19:59:40.256733 6869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Bootstrap starting.
I20260430 19:59:40.288889 6110 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35031
I20260430 19:59:40.289345 6110 heartbeater.cc:499] Master 127.3.190.61:35031 was elected leader, sending a full tablet report...
I20260430 19:59:40.291453 6869 log.cc:826] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Log is configured to *not* fsync() on all Append() calls
I20260430 19:59:40.384079 6869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Bootstrap replayed 1/1 log segments. Stats: ops{read=38 overwritten=1 applied=37 ignored=0} inserts{seen=20 ignored=0} mutations{seen=34 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260430 19:59:40.384836 6869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: Bootstrap complete.
I20260430 19:59:40.403270 6869 raft_consensus.cc:359] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:40.405428 6869 raft_consensus.cc:740] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2878444c3a64840b53f8d5f6fc6211d, State: Initialized, Role: FOLLOWER
I20260430 19:59:40.406376 6869 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 37, Last appended: 6.37, Last appended by leader: 37, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:40.408630 6876 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:40.409197 6876 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:40.409745 6869 sys_catalog.cc:565] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: configured and running, proceeding with master startup.
I20260430 19:59:40.442082 6891 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:40.442404 6891 catalog_manager.cc:1562] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: loading cluster ID for follower catalog manager: success
I20260430 19:59:40.449568 6891 catalog_manager.cc:1584] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: acquiring CA information for follower catalog manager: success
I20260430 19:59:40.456745 6891 catalog_manager.cc:1612] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260430 19:59:40.438654 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 19:59:40.462396 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:59:40.510067 6473 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 19:59:40.510499 6473 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:59:40.547325 6260 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35031
I20260430 19:59:40.547895 6405 heartbeater.cc:344] Connected to a master server at 127.3.190.61:35031
I20260430 19:59:40.555857 6834 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" instance_seqno: 1777579170715407) as {username='slave'} at 127.3.190.3:40353; Asking this server to re-register.
I20260430 19:59:40.556845 6833 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "787275a4c5c64a6aa81be209814867bf" instance_seqno: 1777579168472350) as {username='slave'} at 127.3.190.2:45355; Asking this server to re-register.
I20260430 19:59:40.559221 6405 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:40.559268 6260 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:40.563580 6833 ts_manager.cc:194] Registered new tserver with Master: cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:40.564589 6832 ts_manager.cc:194] Registered new tserver with Master: 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
I20260430 19:59:40.765336 6899 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2878444c3a64840b53f8d5f6fc6211d)
I20260430 19:59:40.765815 6899 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:40.767544 6899 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
I20260430 19:59:40.785658 6844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d" is_pre_election: true
I20260430 19:59:40.786691 6844 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 867d1f5512324df489edc5a61b79df54 in term 6.
W20260430 19:59:40.787528 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 19:59:40.788424 6585 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters: f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:40.789166 6899 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260430 19:59:40.789503 6899 raft_consensus.cc:493] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f2878444c3a64840b53f8d5f6fc6211d)
I20260430 19:59:40.789860 6899 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 6 FOLLOWER]: Advancing to term 7
I20260430 19:59:40.795677 6110 heartbeater.cc:499] Master 127.3.190.61:35031 was elected leader, sending a full tablet report...
I20260430 19:59:40.796803 6899 raft_consensus.cc:515] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:40.799649 6899 leader_election.cc:290] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 election: Requested vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), f2878444c3a64840b53f8d5f6fc6211d (127.3.190.61:35031)
I20260430 19:59:40.800176 6834 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "499a4336efe645d4926c94600c732b82" instance_seqno: 1777579166108385) as {username='slave'} at 127.3.190.1:58077; Asking this server to re-register.
I20260430 19:59:40.801503 6844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "867d1f5512324df489edc5a61b79df54" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: false dest_uuid: "f2878444c3a64840b53f8d5f6fc6211d"
I20260430 19:59:40.802079 6110 heartbeater.cc:461] Registering TS with master...
I20260430 19:59:40.802037 6844 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 6 FOLLOWER]: Advancing to term 7
I20260430 19:59:40.812542 6844 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 867d1f5512324df489edc5a61b79df54 in term 7.
I20260430 19:59:40.815451 6585 leader_election.cc:304] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867d1f5512324df489edc5a61b79df54, f2878444c3a64840b53f8d5f6fc6211d; no voters:
I20260430 19:59:40.816234 6899 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 7 FOLLOWER]: Leader election won for term 7
W20260430 19:59:40.817646 6585 leader_election.cc:336] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 19:59:40.821686 6834 ts_manager.cc:194] Registered new tserver with Master: 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861)
I20260430 19:59:40.823148 6899 raft_consensus.cc:697] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [term 7 LEADER]: Becoming Leader. State: Replica: 867d1f5512324df489edc5a61b79df54, State: Running, Role: LEADER
I20260430 19:59:40.824143 6899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.37, Last appended by leader: 37, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:40.827997 6902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 7 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:40.828608 6902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:40.830024 6904 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260430 19:59:40.835587 6904 catalog_manager.cc:679] Loaded metadata for table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]
I20260430 19:59:40.836815 6904 catalog_manager.cc:679] Loaded metadata for table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb]
I20260430 19:59:40.837946 6904 catalog_manager.cc:679] Loaded metadata for table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]
I20260430 19:59:40.843689 6904 tablet_loader.cc:96] loaded metadata for tablet 87720d2b47d6419bb1420c6f0187d74b (table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d])
I20260430 19:59:40.845758 6904 tablet_loader.cc:96] loaded metadata for tablet d00366d491b247468adcee8fbf41f0a1 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d])
I20260430 19:59:40.847020 6904 tablet_loader.cc:96] loaded metadata for tablet f331ae400bff46f49524f28b87e006ad (table default.table_5ef46544dc424dea9756db410f1a18c1 [id=e932eb2d012c43ff9471d92eb87f0bfb])
I20260430 19:59:40.847975 6904 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260430 19:59:40.851395 6904 catalog_manager.cc:1269] Loaded cluster ID: 4963134f6a694793a4115ae1c2aa919c
I20260430 19:59:40.851598 6904 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260430 19:59:40.855059 6904 catalog_manager.cc:1514] Loading token signing keys...
I20260430 19:59:40.858106 6904 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Loaded TSK: 0
I20260430 19:59:40.859118 6904 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260430 19:59:40.862111 6904 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260430 19:59:40.865152 6904 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260430 19:59:40.877835 6844 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Refusing update from remote peer 867d1f5512324df489edc5a61b79df54: Log matching property violated. Preceding OpId in replica: term: 6 index: 37. Preceding OpId from leader: term: 7 index: 39. (index mismatch)
I20260430 19:59:40.879284 6902 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
W20260430 19:59:40.880722 6585 consensus_peers.cc:597] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 -> Peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Couldn't send request to peer f46e27ec79e44f329de14da21f60e60f. Status: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260430 19:59:40.893630 6876 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 867d1f5512324df489edc5a61b79df54. Latest consensus state: current_term: 7 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:40.894268 6876 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:40.897043 6899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:40.898339 6899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54 [sys.catalog]: This master's current role is: LEADER
I20260430 19:59:40.898986 6906 mvcc.cc:204] Tried to move back new op lower bound from 7280964324862685184 to 7280964324662697984. Current Snapshot: MvccSnapshot[applied={T|T < 7280964324862685184}]
I20260430 19:59:40.901010 6876 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "867d1f5512324df489edc5a61b79df54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } } }
I20260430 19:59:40.901733 6876 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [sys.catalog]: This master's current role is: FOLLOWER
I20260430 19:59:40.940426 6601 master_service.cc:954] ReplaceTablet: received request to replace tablet 87720d2b47d6419bb1420c6f0187d74b from {username='slave'} at 127.0.0.1:57726
W20260430 19:59:40.944492 6600 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:40.949903 6599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57726:
table { table_name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7" } modify_external_catalogs: true
I20260430 19:59:40.950704 6599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57726:
table { table_name: "default.Table_951f1f9e5b9c4759bff2c88e5a4444e7" } modify_external_catalogs: true
I20260430 19:59:40.953092 6597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57726:
name: "default.Table_955554de498d41cebb30bea37e154c53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:40.954568 6598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57726:
table { table_name: "default.Table_5ef46544dc424dea9756db410f1a18c1" } new_table_name: "default.Table_83c2669f3f9548348584e0ecd6d59c9d" modify_external_catalogs: true
I20260430 19:59:40.952179 6600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57726:
table { table_name: "default.Table_a4a99cd76d004be8a0e4b66d9e74fa77" } new_table_name: "default.Table_8bf37998afb24f6db71c7e78a44337dc" modify_external_catalogs: true
I20260430 19:59:40.957286 6791 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:40.958351 6791 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:40.958226 6596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57726:
name: "default.Table_c9b777e92e10430b8f8d1920f4ffc8bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260430 19:59:40.960932 6595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57726:
name: "default.Table_8868e9393dc8483eb17d875dc93821ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260430 19:59:40.961611 6659 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1777579171, eventType=CREATE_TABLE, dbName=default, tableName=table_a4a99cd76d004be8a0e4b66d9e74fa77, message={"server":"","servicePrincipal":"","db":"default","table":"table_a4a99cd76d004be8a0e4b66d9e74fa77","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_a4a99cd76d004be8a0e4b66d9e74fa77\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777579171},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/warehouse/table_a4a99cd76d004be8a0e4b66d9e74fa77\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_a4a99cd76d004be8a0e4b66d9e74fa77\",\"kudu.table_id\":\"f29979fa635848d98e8466f0a3576a6d\",\"transient_lastDdlTime\":\"1777579171\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4963134f6a694793a4115ae1c2aa919c\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777579171,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777579171, eventType=CREATE_TABLE, dbName=default, tableName=table_1f40269e3dc346979445899c9c93ac54, message={"server":"","servicePrincipal":"","db":"default","table":"table_1f40269e3dc346979445899c9c93ac54","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_1f40269e3dc346979445899c9c93ac54\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777579171},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/warehouse/table_1f40269e3dc346979445899c9c93ac54\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_1f40269e3dc346979445899c9c93ac54\",\"kudu.table_id\":\"7a6137ba6a5046ec945d466836f35e0a\",\"transient_lastDdlTime\":\"1777579171\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4963134f6a694793a4115ae1c2aa919c\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777579171,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260430 19:59:40.962297 6594 master_service.cc:954] ReplaceTablet: received request to replace tablet d00366d491b247468adcee8fbf41f0a1 from {username='slave'} at 127.0.0.1:57726
I20260430 19:59:40.965587 6593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57726:
name: "default.Table_b2b7ae30e3404fe8be26c7cc54e7d102"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260430 19:59:40.976200 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:40.978632 6601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Sending DeleteTablet for 3 replicas of tablet 87720d2b47d6419bb1420c6f0187d74b
W20260430 19:59:40.985845 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:40.986263 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:40.987491 6791 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:40.988579 6791 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:40.988878 6594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 867d1f5512324df489edc5a61b79df54: Sending DeleteTablet for 3 replicas of tablet d00366d491b247468adcee8fbf41f0a1
I20260430 19:59:40.994998 6601 catalog_manager.cc:6741] ReplaceTablet: tablet 87720d2b47d6419bb1420c6f0187d74b deleted and replaced by tablet 463116aaa4994f90b9b4e596bbaba13f
I20260430 19:59:41.004125 6594 catalog_manager.cc:6741] ReplaceTablet: tablet d00366d491b247468adcee8fbf41f0a1 deleted and replaced by tablet e0342c4ec4344d7eb80015ccea57d6f7
@ 0x7fc73362e950 google::LogMessage::Flush() at ??:0
I20260430 19:59:41.049866 6194 tablet_service.cc:1558] Processing DeleteTablet for tablet 87720d2b47d6419bb1420c6f0187d74b with delete_type TABLET_DATA_DELETED (replaced by tablet 463116aaa4994f90b9b4e596bbaba13f) from {username='slave'} at 127.0.0.1:47382
I20260430 19:59:41.050179 6192 tablet_service.cc:1558] Processing DeleteTablet for tablet d00366d491b247468adcee8fbf41f0a1 with delete_type TABLET_DATA_DELETED (replaced by tablet e0342c4ec4344d7eb80015ccea57d6f7) from {username='slave'} at 127.0.0.1:47382
@ 0x7fc73362e6bf google::LogMessage::~LogMessage() at ??:0
I20260430 19:59:41.055169 6919 tablet_replica.cc:333] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:41.055944 6919 raft_consensus.cc:2243] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:41.056466 6919 raft_consensus.cc:2272] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:41.058521 6919 ts_tablet_manager.cc:1916] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:41.074558 6919 ts_tablet_manager.cc:1929] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:41.074949 6919 log.cc:1199] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/87720d2b47d6419bb1420c6f0187d74b
I20260430 19:59:41.076079 6919 ts_tablet_manager.cc:1950] T 87720d2b47d6419bb1420c6f0187d74b P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:41.078642 6919 tablet_replica.cc:333] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: stopping tablet replica
I20260430 19:59:41.079049 6588 catalog_manager.cc:5002] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): tablet 87720d2b47d6419bb1420c6f0187d74b (table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]) successfully deleted
I20260430 19:59:41.079357 6919 raft_consensus.cc:2243] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:41.079854 6919 raft_consensus.cc:2272] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:41.081661 6919 ts_tablet_manager.cc:1916] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:41.093320 6919 ts_tablet_manager.cc:1929] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:41.093691 6919 log.cc:1199] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-1/wal/wals/d00366d491b247468adcee8fbf41f0a1
I20260430 19:59:41.094323 6919 ts_tablet_manager.cc:1950] T d00366d491b247468adcee8fbf41f0a1 P 787275a4c5c64a6aa81be209814867bf: Deleting consensus metadata
I20260430 19:59:41.095249 6404 heartbeater.cc:499] Master 127.3.190.60:45507 was elected leader, sending a full tablet report...
I20260430 19:59:41.097040 6588 catalog_manager.cc:5002] TS 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): tablet d00366d491b247468adcee8fbf41f0a1 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
W20260430 19:59:41.105080 6594 catalog_manager.cc:5504] Ignoring report from unknown tablet d28c8f73e1bd4acd8e30805e34662440
W20260430 19:59:41.105381 6594 catalog_manager.cc:5504] Ignoring report from unknown tablet fcb984dd833a443eac422b0821f4fa77
W20260430 19:59:41.105604 6594 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:41.117501 6194 tablet_service.cc:1511] Processing CreateTablet for tablet e0342c4ec4344d7eb80015ccea57d6f7 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:41.118566 6194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0342c4ec4344d7eb80015ccea57d6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:41.122293 6192 tablet_service.cc:1511] Processing CreateTablet for tablet 463116aaa4994f90b9b4e596bbaba13f (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:41.123381 6192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463116aaa4994f90b9b4e596bbaba13f. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fc73e6e52bc kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260430 19:59:41.125833 6338 tablet_service.cc:1558] Processing DeleteTablet for tablet 87720d2b47d6419bb1420c6f0187d74b with delete_type TABLET_DATA_DELETED (replaced by tablet 463116aaa4994f90b9b4e596bbaba13f) from {username='slave'} at 127.0.0.1:53012
I20260430 19:59:41.126384 6043 tablet_service.cc:1558] Processing DeleteTablet for tablet d00366d491b247468adcee8fbf41f0a1 with delete_type TABLET_DATA_DELETED (replaced by tablet e0342c4ec4344d7eb80015ccea57d6f7) from {username='slave'} at 127.0.0.1:54274
I20260430 19:59:41.126915 6339 tablet_service.cc:1558] Processing DeleteTablet for tablet d00366d491b247468adcee8fbf41f0a1 with delete_type TABLET_DATA_DELETED (replaced by tablet e0342c4ec4344d7eb80015ccea57d6f7) from {username='slave'} at 127.0.0.1:53012
I20260430 19:59:41.127470 6044 tablet_service.cc:1558] Processing DeleteTablet for tablet 87720d2b47d6419bb1420c6f0187d74b with delete_type TABLET_DATA_DELETED (replaced by tablet 463116aaa4994f90b9b4e596bbaba13f) from {username='slave'} at 127.0.0.1:54274
I20260430 19:59:41.127781 6042 tablet_service.cc:1511] Processing CreateTablet for tablet e0342c4ec4344d7eb80015ccea57d6f7 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:41.128201 6337 tablet_service.cc:1511] Processing CreateTablet for tablet e0342c4ec4344d7eb80015ccea57d6f7 (DEFAULT_TABLE table=default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:41.128866 6042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0342c4ec4344d7eb80015ccea57d6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:41.129401 6337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0342c4ec4344d7eb80015ccea57d6f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:41.130947 6041 tablet_service.cc:1511] Processing CreateTablet for tablet 463116aaa4994f90b9b4e596bbaba13f (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:41.131421 6336 tablet_service.cc:1511] Processing CreateTablet for tablet 463116aaa4994f90b9b4e596bbaba13f (DEFAULT_TABLE table=default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260430 19:59:41.132031 6041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463116aaa4994f90b9b4e596bbaba13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:41.131439 6594 catalog_manager.cc:5671] T f331ae400bff46f49524f28b87e006ad P cd13173a6c184acaa26b63bdbf8d6656 reported cstate change: term changed from 0 to 1, leader changed from <none> to 787275a4c5c64a6aa81be209814867bf (127.3.190.2). New cstate: current_term: 1 leader_uuid: "787275a4c5c64a6aa81be209814867bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } }
I20260430 19:59:41.132465 6336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463116aaa4994f90b9b4e596bbaba13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260430 19:59:41.165846 6921 tablet_replica.cc:333] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
I20260430 19:59:41.166160 6922 tablet_replica.cc:333] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:41.166679 6921 raft_consensus.cc:2243] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:59:41.166926 6922 raft_consensus.cc:2243] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 1 LEADER]: Raft consensus shutting down.
I20260430 19:59:41.167913 6922 raft_consensus.cc:2272] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:41.169778 6922 ts_tablet_manager.cc:1916] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:41.175226 6921 raft_consensus.cc:2272] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:41.177376 6921 ts_tablet_manager.cc:1916] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:41.187584 6927 tablet_bootstrap.cc:492] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:41.193336 6926 tablet_bootstrap.cc:492] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
@ 0x7fc73e6e4a7c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260430 19:59:41.197608 6925 tablet_bootstrap.cc:492] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
I20260430 19:59:41.209273 6594 master_service.cc:954] ReplaceTablet: received request to replace tablet fcb984dd833a443eac422b0821f4fa77 from {username='slave'} at 127.0.0.1:57726
I20260430 19:59:41.212675 6922 ts_tablet_manager.cc:1929] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:41.213047 6922 log.cc:1199] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/d00366d491b247468adcee8fbf41f0a1
I20260430 19:59:41.213806 6922 ts_tablet_manager.cc:1950] T d00366d491b247468adcee8fbf41f0a1 P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:41.215561 6587 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet d00366d491b247468adcee8fbf41f0a1 (table default.table_a4a99cd76d004be8a0e4b66d9e74fa77 [id=f29979fa635848d98e8466f0a3576a6d]) successfully deleted
I20260430 19:59:41.216017 6594 master_service.cc:954] ReplaceTablet: received request to replace tablet 87720d2b47d6419bb1420c6f0187d74b from {username='slave'} at 127.0.0.1:57726
@ 0x7fc73e6e8af2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260430 19:59:41.229254 6922 tablet_replica.cc:333] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: stopping tablet replica
I20260430 19:59:41.229982 6922 raft_consensus.cc:2243] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:41.230466 6922 raft_consensus.cc:2272] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:41.239828 6922 ts_tablet_manager.cc:1916] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fc73e6e8aaa _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260430 19:59:41.247704 6925 tablet_bootstrap.cc:654] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:41.249536 6927 tablet_bootstrap.cc:654] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:41.237344 6926 tablet_bootstrap.cc:654] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
@ 0x7fc73e6e8a3a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260430 19:59:41.266259 6922 ts_tablet_manager.cc:1929] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:41.267267 6922 log.cc:1199] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-0/wal/wals/87720d2b47d6419bb1420c6f0187d74b
I20260430 19:59:41.267207 6921 ts_tablet_manager.cc:1929] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:41.267994 6922 ts_tablet_manager.cc:1950] T 87720d2b47d6419bb1420c6f0187d74b P 499a4336efe645d4926c94600c732b82: Deleting consensus metadata
I20260430 19:59:41.270046 6927 tablet_bootstrap.cc:492] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:41.270462 6927 ts_tablet_manager.cc:1403] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.083s user 0.010s sys 0.000s
I20260430 19:59:41.270570 6587 catalog_manager.cc:5002] TS 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): tablet 87720d2b47d6419bb1420c6f0187d74b (table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]) successfully deleted
I20260430 19:59:41.272852 6926 tablet_bootstrap.cc:492] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:41.273253 6926 ts_tablet_manager.cc:1403] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.080s user 0.010s sys 0.000s
I20260430 19:59:41.273114 6927 raft_consensus.cc:359] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.273821 6927 raft_consensus.cc:385] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:41.274097 6927 raft_consensus.cc:740] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:41.275802 6926 raft_consensus.cc:359] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.276461 6926 raft_consensus.cc:385] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:41.276743 6926 raft_consensus.cc:740] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:41.267596 6921 log.cc:1199] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/87720d2b47d6419bb1420c6f0187d74b
I20260430 19:59:41.286964 6925 tablet_bootstrap.cc:492] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:41.287453 6925 ts_tablet_manager.cc:1403] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.090s user 0.005s sys 0.008s
I20260430 19:59:41.286928 6927 consensus_queue.cc:260] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
@ 0x7fc73e6e8a02 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260430 19:59:41.288631 6259 heartbeater.cc:499] Master 127.3.190.60:45507 was elected leader, sending a full tablet report...
I20260430 19:59:41.289289 6927 ts_tablet_manager.cc:1434] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
W20260430 19:59:41.295250 6594 catalog_manager.cc:5504] Ignoring report from unknown tablet fcb984dd833a443eac422b0821f4fa77
W20260430 19:59:41.295511 6594 catalog_manager.cc:5504] Ignoring report from unknown tablet fb30716bb9174641ba41885f792e6364
W20260430 19:59:41.295717 6594 catalog_manager.cc:5504] Ignoring report from unknown tablet 8da8b25a7d17492b8830605a61a2f2ea
I20260430 19:59:41.279692 6921 ts_tablet_manager.cc:1950] T 87720d2b47d6419bb1420c6f0187d74b P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:41.302680 6587 catalog_manager.cc:5002] TS cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): tablet 87720d2b47d6419bb1420c6f0187d74b (table default.table_951f1f9e5b9c4759bff2c88e5a4444e7 [id=8bf2640796ce4d5081894bec93fd773d]) successfully deleted
I20260430 19:59:41.304949 6926 consensus_queue.cc:260] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.290208 6925 raft_consensus.cc:359] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.306716 6925 raft_consensus.cc:385] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:41.307027 6925 raft_consensus.cc:740] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:41.290311 6927 tablet_bootstrap.cc:492] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf: Bootstrap starting.
I20260430 19:59:41.307714 6925 consensus_queue.cc:260] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
@ 0x7fc73e6e7cfe std::__1::__function::__func<>::operator()() at ??:0
@ 0x5b3bc5 std::__1::__function::__value_func<>::operator()() at ??:?
I20260430 19:59:41.320505 6921 tablet_replica.cc:333] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: stopping tablet replica
@ 0x5b39fa std::__1::function<>::operator()() at ??:?
I20260430 19:59:41.321326 6921 raft_consensus.cc:2243] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260430 19:59:41.321862 6921 raft_consensus.cc:2272] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260430 19:59:41.323969 6921 ts_tablet_manager.cc:1916] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260430 19:59:41.328890 6926 ts_tablet_manager.cc:1434] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.055s user 0.007s sys 0.000s
I20260430 19:59:41.330665 6927 tablet_bootstrap.cc:654] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:41.331444 6109 heartbeater.cc:499] Master 127.3.190.60:45507 was elected leader, sending a full tablet report...
W20260430 19:59:41.337580 6594 catalog_manager.cc:5504] Ignoring report from unknown tablet fcb984dd833a443eac422b0821f4fa77
@ 0x7fc735854fbb kudu::Thread::SuperviseThread() at ??:0
@ 0x479510 __tsan_thread_start_func at ??:?
I20260430 19:59:41.344013 6925 ts_tablet_manager.cc:1434] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.056s user 0.006s sys 0.000s
I20260430 19:59:41.353989 6927 tablet_bootstrap.cc:492] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf: No bootstrap required, opened a new log
I20260430 19:59:41.354418 6927 ts_tablet_manager.cc:1403] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.008s
@ 0x7fc736f476db start_thread at ??:0
I20260430 19:59:41.362524 6927 raft_consensus.cc:359] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.363238 6927 raft_consensus.cc:385] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:41.363494 6927 raft_consensus.cc:740] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Initialized, Role: FOLLOWER
I20260430 19:59:41.364118 6927 consensus_queue.cc:260] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.366032 6926 tablet_bootstrap.cc:492] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656: Bootstrap starting.
I20260430 19:59:41.381990 6927 ts_tablet_manager.cc:1434] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf: Time spent starting tablet: real 0.027s user 0.004s sys 0.002s
I20260430 19:59:41.384057 6925 tablet_bootstrap.cc:492] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82: Bootstrap starting.
@ 0x7fc7322d361f clone at ??:0
I20260430 19:59:41.394201 6926 tablet_bootstrap.cc:654] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:41.400947 6925 tablet_bootstrap.cc:654] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82: Neither blocks nor log segments found. Creating new log.
I20260430 19:59:41.432619 6926 tablet_bootstrap.cc:492] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656: No bootstrap required, opened a new log
I20260430 19:59:41.433167 6926 ts_tablet_manager.cc:1403] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656: Time spent bootstrapping tablet: real 0.069s user 0.011s sys 0.000s
I20260430 19:59:41.433260 6921 ts_tablet_manager.cc:1929] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260430 19:59:41.433744 6921 log.cc:1199] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: Deleting WAL directory at /tmp/dist-test-task07fIBA/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1777579106062754-3832-0/minicluster-data/ts-2/wal/wals/d00366d491b247468adcee8fbf41f0a1
I20260430 19:59:41.434571 6921 ts_tablet_manager.cc:1950] T d00366d491b247468adcee8fbf41f0a1 P cd13173a6c184acaa26b63bdbf8d6656: Deleting consensus metadata
I20260430 19:59:41.435920 6926 raft_consensus.cc:359] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.436653 6926 raft_consensus.cc:385] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:41.436959 6926 raft_consensus.cc:740] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd13173a6c184acaa26b63bdbf8d6656, State: Initialized, Role: FOLLOWER
I20260430 19:59:41.437675 6926 consensus_queue.cc:260] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.444007 6925 tablet_bootstrap.cc:492] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82: No bootstrap required, opened a new log
I20260430 19:59:41.444564 6925 ts_tablet_manager.cc:1403] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82: Time spent bootstrapping tablet: real 0.061s user 0.006s sys 0.007s
I20260430 19:59:41.445385 6910 raft_consensus.cc:493] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:41.445858 6910 raft_consensus.cc:515] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.447261 6926 ts_tablet_manager.cc:1434] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656: Time spent starting tablet: real 0.014s user 0.001s sys 0.005s
I20260430 19:59:41.447957 6910 leader_election.cc:290] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:41.447834 6925 raft_consensus.cc:359] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.448490 6925 raft_consensus.cc:385] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260430 19:59:41.448774 6925 raft_consensus.cc:740] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499a4336efe645d4926c94600c732b82, State: Initialized, Role: FOLLOWER
I20260430 19:59:41.450095 6063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0342c4ec4344d7eb80015ccea57d6f7" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:41.449920 6925 consensus_queue.cc:260] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.450799 6063 raft_consensus.cc:2468] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:41.451793 6358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0342c4ec4344d7eb80015ccea57d6f7" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
I20260430 19:59:41.452422 6358 raft_consensus.cc:2468] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:41.454123 6149 leader_election.cc:304] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:41.455721 6910 raft_consensus.cc:2804] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:41.456138 6910 raft_consensus.cc:493] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:41.456518 6910 raft_consensus.cc:3060] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:41.462989 6910 raft_consensus.cc:515] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.465677 6063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0342c4ec4344d7eb80015ccea57d6f7" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82"
I20260430 19:59:41.466270 6063 raft_consensus.cc:3060] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:41.468111 6925 ts_tablet_manager.cc:1434] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20260430 19:59:41.472358 6063 raft_consensus.cc:2468] T e0342c4ec4344d7eb80015ccea57d6f7 P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:41.473635 6149 leader_election.cc:304] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:41.474542 6911 raft_consensus.cc:2804] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:41.475041 6911 raft_consensus.cc:697] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [term 1 LEADER]: Becoming Leader. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Running, Role: LEADER
I20260430 19:59:41.475806 6911 consensus_queue.cc:237] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.477147 6358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0342c4ec4344d7eb80015ccea57d6f7" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656"
I20260430 19:59:41.477685 6358 raft_consensus.cc:3060] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:41.483636 6910 leader_election.cc:290] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Requested vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:41.484448 6358 raft_consensus.cc:2468] T e0342c4ec4344d7eb80015ccea57d6f7 P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:41.503983 6910 raft_consensus.cc:493] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:41.504470 6910 raft_consensus.cc:515] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.506440 6910 leader_election.cc:290] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:41.509052 6063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463116aaa4994f90b9b4e596bbaba13f" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82" is_pre_election: true
I20260430 19:59:41.511018 6358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463116aaa4994f90b9b4e596bbaba13f" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656" is_pre_election: true
I20260430 19:59:41.510550 6063 raft_consensus.cc:2468] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:41.512424 6149 leader_election.cc:304] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:41.513590 6358 raft_consensus.cc:2468] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 0.
I20260430 19:59:41.514968 6910 raft_consensus.cc:2804] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260430 19:59:41.515386 6910 raft_consensus.cc:493] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260430 19:59:41.515748 6910 raft_consensus.cc:3060] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:41.523245 6910 raft_consensus.cc:515] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:41.525223 6910 leader_election.cc:290] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Requested vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
I20260430 19:59:41.526484 6063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463116aaa4994f90b9b4e596bbaba13f" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499a4336efe645d4926c94600c732b82"
I20260430 19:59:41.526542 6358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463116aaa4994f90b9b4e596bbaba13f" candidate_uuid: "787275a4c5c64a6aa81be209814867bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd13173a6c184acaa26b63bdbf8d6656"
I20260430 19:59:41.527091 6063 raft_consensus.cc:3060] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:41.527143 6358 raft_consensus.cc:3060] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Advancing to term 1
I20260430 19:59:41.533823 6063 raft_consensus.cc:2468] T 463116aaa4994f90b9b4e596bbaba13f P 499a4336efe645d4926c94600c732b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:41.535040 6149 leader_election.cc:304] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf; no voters:
I20260430 19:59:41.536007 6358 raft_consensus.cc:2468] T 463116aaa4994f90b9b4e596bbaba13f P cd13173a6c184acaa26b63bdbf8d6656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787275a4c5c64a6aa81be209814867bf in term 1.
I20260430 19:59:41.536152 6910 raft_consensus.cc:2804] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 1 FOLLOWER]: Leader election won for term 1
I20260430 19:59:41.540062 6910 raft_consensus.cc:697] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [term 1 LEADER]: Becoming Leader. State: Replica: 787275a4c5c64a6aa81be209814867bf, State: Running, Role: LEADER
I20260430 19:59:41.540885 6910 consensus_queue.cc:237] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
W20260430 19:59:41.660432 6292 connection.cc:570] server connection from 127.0.0.1:53012 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:41.661109 5999 connection.cc:570] client connection to 127.3.190.60:45507 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:41.663712 6416 connection.cc:570] client connection to 127.3.190.60:45507 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:41.665344 6416 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.190.60:45507)
W20260430 19:59:41.665977 6416 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.190.60:45507)
W20260430 19:59:41.667742 6416 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.190.60:45507)
W20260430 19:59:41.668543 6294 connection.cc:570] client connection to 127.3.190.60:45507 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260430 19:59:41.670717 6933 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:59:41.663835 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.60:45507 (local address 127.3.190.1:38995)
0430 19:59:41.664787 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0430 19:59:41.664833 (+ 46us) client_negotiation.cc:175] Beginning negotiation
0430 19:59:41.665089 (+ 256us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0430 19:59:41.669964 (+ 4875us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":307}
W20260430 19:59:41.671123 6416 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.190.60:45507)
W20260430 19:59:41.673025 6895 negotiation.cc:336] Failed RPC negotiation. Trace:
0430 19:59:41.665929 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.3.190.60:45507 (local address 127.3.190.2:40969)
0430 19:59:41.666815 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0430 19:59:41.666858 (+ 43us) client_negotiation.cc:175] Beginning negotiation
0430 19:59:41.667087 (+ 229us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0430 19:59:41.672200 (+ 5113us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":284}
I20260430 19:59:41.696421 6948 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:41.696914 6948 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:41.700520 6948 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:41.702814 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:41.706290 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:41.707567 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:41.707983 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:41.708702 6948 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 19:59:42.287389 6954 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:42.287885 6954 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:42.289325 6954 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 19:59:42.290992 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 19:59:42.291523 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:42.291805 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 19:59:42.292358 6954 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:42.372042 6955 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:42.372421 6955 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:42.373997 6955 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:42.388960 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:42.389391 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:42.389734 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:42.390297 6955 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 19:59:42.437839 6952 consensus_queue.cc:1048] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:42.457243 6952 consensus_queue.cc:1048] T 463116aaa4994f90b9b4e596bbaba13f P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:42.678942 3832 master-stress-test.cc:577] Killing test thread 1/10
I20260430 19:59:42.702266 6911 consensus_queue.cc:1048] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:42.705763 6911 consensus_queue.cc:1048] T e0342c4ec4344d7eb80015ccea57d6f7 P 787275a4c5c64a6aa81be209814867bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260430 19:59:43.205650 6971 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:43.206056 6971 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:43.207559 6971 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:43.212530 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:43.213047 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:43.213385 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:43.214054 6971 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 19:59:43.761307 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 6: this message will repeat every 5th retry.
I20260430 19:59:44.346822 6978 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:44.347270 6978 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:44.348875 6978 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:44.353289 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:44.353801 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:44.354152 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:44.354815 6978 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 19:59:44.566529 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000703c0 after lost signal to thread 6770
W20260430 19:59:44.567222 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9820 after lost signal to thread 6868
I20260430 19:59:45.502404 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 19:59:45.502902 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:59:46.336898 6994 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:46.337363 6994 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:46.338804 6994 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:46.343263 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:46.343626 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:46.343931 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:46.344568 6994 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 19:59:46.435624 6991 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:46.436050 6991 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:46.437469 6991 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 19:59:46.439201 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 19:59:46.439682 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:46.439944 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:46.440445 6991 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 19:59:49.106432 7018 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:49.107070 7018 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:49.108757 7018 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:49.111538 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20260430 19:59:49.114672 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 19:59:49.115110 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 19:59:49.115435 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:49.116024 7018 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 19:59:50.506783 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 19:59:50.507346 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:59:51.259388 7042 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:51.259959 7042 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 19:59:51.262081 7042 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 19:59:51.263732 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 19:59:51.264247 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 19:59:51.264611 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 19:59:51.265183 7042 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 19:59:53.461673 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 11: this message will repeat every 5th retry.
I20260430 19:59:53.462038 7066 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 19:59:53.462697 7066 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 19:59:53.464602 7066 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 19:59:53.470402 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 19:59:53.470875 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 19:59:53.471175 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 19:59:53.471881 7066 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 19:59:55.510622 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 19:59:55.511140 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 19:59:59.855686 7141 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 19:59:59.856281 7141 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 19:59:59.858476 7141 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 19:59:59.860267 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 19:59:59.860756 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 19:59:59.861008 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 19:59:59.861533 7141 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:00:00.514851 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:00.515352 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:00.883353 7162 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:00:00.883882 7162 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:00:00.885761 7162 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:00:00.887398 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 2 similar messages]
W20260430 20:00:00.890686 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:00:00.891233 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:00:00.891510 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:00:00.892100 7162 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:00:02.451745 7187 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:02.452260 7187 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:02.453830 7187 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:02.455504 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:02.456116 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:02.456398 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:02.456943 7187 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:00:03.011706 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 16: this message will repeat every 5th retry.
I20260430 20:00:05.518392 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:05.518867 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:08.484750 7271 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:08.485138 7271 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:08.486974 7271 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:08.488428 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:08.488938 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:08.489274 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:08.489861 7271 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:00:09.285472 7282 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:00:09.286026 7282 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:00:09.287776 7282 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:00:09.290297 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:00:09.293444 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:00:09.293867 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:00:09.294173 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:00:09.294800 7282 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:00:10.521803 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:10.522188 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:11.106938 7305 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:11.107522 7305 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:00:11.109103 7305 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:00:11.110685 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:00:11.111217 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:00:11.111480 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:00:11.112025 7305 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:00:12.772447 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 21: this message will repeat every 5th retry.
I20260430 20:00:13.721244 7344 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:00:13.721611 7344 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:00:13.723049 7344 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:00:13.728205 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:00:13.728645 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:00:13.728973 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:00:13.729529 7344 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:00:15.525281 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:15.525628 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:20.529141 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:20.529657 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:00:23.850871 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 26: this message will repeat every 5th retry.
I20260430 20:00:24.631579 7502 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:24.632129 7502 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:24.634274 7502 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:24.636202 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:24.636713 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:24.637060 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:24.637588 7502 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:00:25.532521 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:25.532971 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:26.575387 7521 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:26.575920 7521 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:00:26.578171 7521 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:00:26.579417 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:00:26.579908 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:00:26.580170 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:00:26.580739 7521 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:00:30.536887 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:30.537474 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:30.858510 7575 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:30.859164 7575 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:00:30.860903 7575 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:00:30.862367 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:00:30.862916 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:00:30.863168 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:00:30.863722 7575 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:00:31.417936 6259 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (128 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:00:31.519217 6109 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (103 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:00:31.549840 6407 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:35659 (136 consecutive failures): Network error: Failed to ping master at 127.3.190.62:35659: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:00:31.946564 6400 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8d60 after lost signal to thread 6271
W20260430 20:00:31.947531 6400 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3ca0 after lost signal to thread 6403
W20260430 20:00:32.809320 6255 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5640 after lost signal to thread 6124
W20260430 20:00:32.810227 6255 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3760 after lost signal to thread 6258
I20260430 20:00:33.666129 7612 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:00:33.666610 7612 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:00:33.668550 7612 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:00:33.671017 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260430 20:00:33.674515 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:00:33.675180 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:00:33.675690 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:00:33.676472 7612 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 20:00:34.213975 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 31: this message will repeat every 5th retry.
I20260430 20:00:34.915891 7624 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:00:34.916308 7624 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:00:34.918079 7624 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:00:34.922669 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:00:34.923235 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:00:34.923663 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:00:34.924353 7624 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:00:35.540508 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:35.540956 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:36.480404 7647 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:36.480902 7647 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:36.482501 7647 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:36.484160 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:36.484654 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:36.484965 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:36.485479 7647 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:00:40.544904 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:40.545334 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:41.599117 7717 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:41.599731 7717 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:41.601706 7717 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:41.603279 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:41.603768 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:41.604107 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:41.604604 7717 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:00:44.918959 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 36: this message will repeat every 5th retry.
I20260430 20:00:45.548202 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:45.548657 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:48.265544 7803 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:48.266139 7803 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:00:48.268294 7803 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:00:48.269919 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:00:48.270423 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:00:48.270684 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:00:48.271215 7803 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:00:48.540550 7813 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:48.541065 7813 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:48.542969 7813 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:48.544317 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:48.544975 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:48.545271 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:48.545748 7813 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:00:50.142196 7830 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:00:50.142798 7830 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:00:50.144986 7830 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:00:50.147655 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260430 20:00:50.151011 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:00:50.151517 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:00:50.151837 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:00:50.152875 7830 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:00:50.551543 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:50.552078 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:00:53.112257 7855 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:53.112785 7855 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:53.114869 7855 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:53.116379 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:53.116945 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:53.117336 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:53.118003 7855 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:00:53.875346 6255 debug-util.cc:398] Leaking SignalData structure 0x7b08000d4ac0 after lost signal to thread 6124
W20260430 20:00:53.876242 6255 debug-util.cc:398] Leaking SignalData structure 0x7b080006d780 after lost signal to thread 6258
W20260430 20:00:55.345624 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 41: this message will repeat every 5th retry.
I20260430 20:00:55.555004 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:00:55.555371 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:00:58.446556 6149 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.190.3:45977, user_credentials={real_user=slave}} blocked reactor thread for 67752.2us
W20260430 20:00:59.265308 6105 debug-util.cc:398] Leaking SignalData structure 0x7b08000dca60 after lost signal to thread 5960
W20260430 20:00:59.266335 6105 debug-util.cc:398] Leaking SignalData structure 0x7b08000b2100 after lost signal to thread 6108
I20260430 20:00:59.385767 7908 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:00:59.386437 7908 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:00:59.388674 7908 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:00:59.390219 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:00:59.390883 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:00:59.391146 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:00:59.391675 7908 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:00.559195 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:00.559716 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:00.571837 7925 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:01:00.572396 7925 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:01:00.573957 7925 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:01:00.575695 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:01:00.579181 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:01:00.579735 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:01:00.580047 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:01:00.580634 7925 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:01:01.490847 7931 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:01.491480 7931 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:01:01.492997 7931 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:01:01.494381 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:01:01.495051 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:01:01.495340 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:01:01.495838 7931 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:01:05.249854 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 46: this message will repeat every 5th retry.
I20260430 20:01:05.562635 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:05.563004 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:07.531836 8012 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:07.532511 8012 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:01:07.534869 8012 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:01:07.536316 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:01:07.536830 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:01:07.537096 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:01:07.537654 8012 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:10.566720 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:10.567242 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:14.832391 8107 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:01:14.832870 8107 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:01:14.834460 8107 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:01:14.836612 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:01:14.839393 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:01:14.839859 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:01:14.840128 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:01:14.840778 8107 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 20:01:15.195412 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 51: this message will repeat every 5th retry.
I20260430 20:01:15.570065 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:15.570502 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:16.834530 8129 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:16.835062 8129 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:01:16.836855 8129 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:01:16.838413 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:01:16.839217 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:01:16.839522 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:01:16.840065 8129 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:19.537220 8147 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:19.537930 8147 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:01:19.540326 8147 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:01:19.543471 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:01:19.544378 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:01:19.544703 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:01:19.545593 8167 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:20.342092 8175 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:20.342715 8175 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:01:20.344470 8175 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:01:20.346141 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:01:20.346904 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:01:20.347316 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:01:20.348019 8175 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:20.574311 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:20.574808 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:25.530222 8240 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:25.530654 8240 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:01:25.532666 8240 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:01:25.534328 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:01:25.534837 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:01:25.535091 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:01:25.535648 8240 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:25.577572 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:25.578003 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:01:25.811785 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 56: this message will repeat every 5th retry.
I20260430 20:01:26.314550 8250 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:01:26.315335 8250 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:01:26.320020 8250 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:01:26.321938 6820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:01:26.327062 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:01:26.327749 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:01:26.328094 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:01:26.328855 8250 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:01:30.581481 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:30.581921 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:31.175621 8297 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:31.176230 8297 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:01:31.177978 8297 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:01:31.180070 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:01:31.183576 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:01:31.183964 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:01:31.184903 8297 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:01:31.462829 6263 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:35659 (315 consecutive failures): Network error: Failed to ping master at 127.3.190.62:35659: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:01:31.946954 6111 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:35659 (254 consecutive failures): Network error: Failed to ping master at 127.3.190.62:35659: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:01:31.947609 6407 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:35659 (256 consecutive failures): Network error: Failed to ping master at 127.3.190.62:35659: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:01:35.127549 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1560 after lost signal to thread 6770
W20260430 20:01:35.128073 6865 debug-util.cc:398] Leaking SignalData structure 0x7b080007b160 after lost signal to thread 6868
W20260430 20:01:35.405471 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 61: this message will repeat every 5th retry.
I20260430 20:01:35.611079 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:35.611522 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:37.613669 8364 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:01:37.614188 8364 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:01:37.615759 8364 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:01:37.618089 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:01:37.620916 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:01:37.621515 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:01:37.621795 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:01:37.622483 8364 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:01:40.470072 6823 maintenance_manager.cc:419] P f2878444c3a64840b53f8d5f6fc6211d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033350
I20260430 20:01:40.544888 6822 maintenance_manager.cc:643] P f2878444c3a64840b53f8d5f6fc6211d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.074s user 0.048s sys 0.009s Metrics: {"bytes_written":12396,"cfile_init":1,"drs_written":1,"lbm_read_time_us":325,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2217,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":22,"thread_start_us":445,"threads_started":1,"wal-append.queue_time_us":731}
I20260430 20:01:40.546980 6823 maintenance_manager.cc:419] P f2878444c3a64840b53f8d5f6fc6211d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1203 bytes on disk
I20260430 20:01:40.548439 6822 maintenance_manager.cc:643] P f2878444c3a64840b53f8d5f6fc6211d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":221,"lbm_reads_lt_1ms":4}
I20260430 20:01:40.615056 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:40.615517 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:41.515354 8409 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:41.515938 8409 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:01:41.518157 8409 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:01:41.519805 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:01:41.520326 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:01:41.520555 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:01:41.521165 8409 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:01:45.433365 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 66: this message will repeat every 5th retry.
I20260430 20:01:45.618292 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:45.618723 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:47.824923 8511 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:47.825603 8511 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:01:47.827898 8511 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:01:47.829352 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:01:47.830132 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:01:47.830454 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:01:47.831003 8511 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:49.698506 8537 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:01:49.699132 8537 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:01:49.700717 8537 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:01:49.703217 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:01:49.706521 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:01:49.707062 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:01:49.707341 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:01:49.707909 8537 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:01:50.018121 8544 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:50.018726 8544 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:01:50.020426 8544 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:01:50.022086 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:01:50.022645 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:01:50.022897 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:01:50.023406 8544 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:01:50.621558 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:50.621960 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:01:54.741988 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 71: this message will repeat every 5th retry.
I20260430 20:01:55.342361 8638 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:01:55.342921 8638 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:01:55.344432 8638 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:01:55.346414 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:01:55.349488 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:01:55.349957 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:01:55.350268 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:01:55.350873 8638 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:01:55.624317 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:01:55.624698 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:01:57.846415 8676 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:01:57.846967 8676 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:01:57.848522 8676 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:01:57.853245 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:01:57.853654 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:01:57.854003 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:01:57.854579 8676 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:01:59.964673 8715 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:01:59.965230 8715 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:01:59.967012 8715 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:01:59.968897 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:01:59.969861 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:01:59.970248 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:01:59.971021 8715 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:00.628575 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:00.629119 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:02:00.677201 6105 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6140 after lost signal to thread 5960
W20260430 20:02:00.678117 6105 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5480 after lost signal to thread 6108
W20260430 20:02:04.909256 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 76: this message will repeat every 5th retry.
I20260430 20:02:05.631927 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:05.632400 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:02:08.299491 8839 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:02:08.299970 8839 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:02:08.301791 8839 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:02:08.304118 6820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260430 20:02:08.307075 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:02:08.307510 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:02:08.307868 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:02:08.308468 8839 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:02:09.670099 8861 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:09.670724 8861 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:02:09.672763 8861 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:02:09.674448 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:02:09.675026 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:02:09.675263 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:02:09.675873 8861 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:10.037561 8872 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:10.038133 8872 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:02:10.040799 8872 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:02:10.042202 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:02:10.042905 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:02:10.043162 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:02:10.043746 8872 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:10.636224 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:10.636633 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:02:14.831601 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 81: this message will repeat every 5th retry.
I20260430 20:02:15.639675 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:15.640038 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:02:17.523876 6400 debug-util.cc:398] Leaking SignalData structure 0x7b080006d4c0 after lost signal to thread 6271
W20260430 20:02:17.524844 6400 debug-util.cc:398] Leaking SignalData structure 0x7b08000de8e0 after lost signal to thread 6403
W20260430 20:02:18.227751 6105 debug-util.cc:398] Leaking SignalData structure 0x7b080003d400 after lost signal to thread 5960
W20260430 20:02:18.228688 6105 debug-util.cc:398] Leaking SignalData structure 0x7b08000d2ba0 after lost signal to thread 6108
I20260430 20:02:20.643725 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:20.644224 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:02:25.104060 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 86: this message will repeat every 5th retry.
I20260430 20:02:25.647243 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:25.647598 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:02:25.773296 9124 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:25.773975 9124 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:02:25.776244 9124 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:02:25.777889 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:02:25.778410 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:02:25.778685 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:02:25.779299 9124 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:27.780591 9171 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:02:27.781097 9171 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:02:27.783167 9171 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:02:27.785440 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:02:27.788127 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:02:27.788517 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:02:27.788800 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:02:27.789371 9171 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:02:27.966403 9181 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:27.966969 9181 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:02:27.970710 9181 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:02:27.973167 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:02:27.974628 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:02:27.974998 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:02:27.976081 9181 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:30.581765 9221 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:02:30.582338 9221 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:02:30.584163 9221 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:02:30.589452 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:02:30.589952 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:02:30.590281 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:02:30.590950 9221 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:02:30.649883 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:30.650341 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:02:31.558254 9226 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:31.559042 9226 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:02:31.561097 9226 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:02:31.562551 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:02:31.563186 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:02:31.563512 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:02:31.564198 9226 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:02:31.801528 6263 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:35659 (463 consecutive failures): Network error: Failed to ping master at 127.3.190.62:35659: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:02:32.344740 6404 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (343 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:02:32.372582 6109 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (343 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:02:34.289368 9283 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:34.289897 9283 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:02:34.291853 9283 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:02:34.293291 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:02:34.293855 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:02:34.294169 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:02:34.294775 9283 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:35.653525 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:35.653954 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:02:35.997092 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 91: this message will repeat every 5th retry.
I20260430 20:02:40.657392 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:40.657979 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:02:41.865383 9413 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:02:41.865845 9413 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:02:41.867609 9413 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:02:41.869772 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260430 20:02:41.872498 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:02:41.872936 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:02:41.873191 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:02:41.873835 9413 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 20:02:42.933804 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8d60 after lost signal to thread 6770
W20260430 20:02:42.934442 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000b98c0 after lost signal to thread 6868
W20260430 20:02:43.960850 6255 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0e40 after lost signal to thread 6124
W20260430 20:02:43.961843 6255 debug-util.cc:398] Leaking SignalData structure 0x7b08000bafc0 after lost signal to thread 6258
I20260430 20:02:45.692451 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:45.692828 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:02:46.596328 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 96: this message will repeat every 5th retry.
I20260430 20:02:48.254524 9491 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:48.255255 9491 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:02:48.257956 9491 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:02:48.259428 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:02:48.260052 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:02:48.260314 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:02:48.260902 9491 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:49.313920 9535 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:49.314729 9535 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:02:49.317111 9535 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:02:49.318756 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:02:49.319299 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:02:49.319700 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:02:49.320283 9535 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:02:50.462821 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9bc0 after lost signal to thread 6770
W20260430 20:02:50.463367 6865 debug-util.cc:398] Leaking SignalData structure 0x7b0800070360 after lost signal to thread 6868
I20260430 20:02:50.701718 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:50.702154 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:02:54.370090 9616 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:02:54.370698 9616 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:02:54.372373 9616 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:02:54.374693 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:02:54.378635 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:02:54.379186 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:02:54.379459 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:02:54.379992 9616 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:02:55.326122 9593 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:02:55.326529 9593 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:02:55.328061 9593 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:02:55.329427 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:02:55.330008 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:02:55.330262 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:02:55.330807 9593 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:02:55.705838 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:02:55.706266 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:02:56.851403 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 101: this message will repeat every 5th retry.
I20260430 20:03:00.167172 9722 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:00.167764 9722 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:03:00.169365 9722 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:03:00.171121 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:03:00.171653 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:03:00.171917 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:03:00.172490 9722 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:03:00.709838 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:00.710320 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:03:01.833840 9748 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:03:01.834372 9748 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:03:01.835996 9748 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:03:01.838728 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:03:01.841467 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:03:01.841869 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:03:01.842183 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:03:01.842784 9748 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:03:05.713080 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:05.713459 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:03:06.240478 9817 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:06.241140 9817 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:03:06.243700 9817 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:03:06.245247 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:03:06.245734 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:03:06.245960 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:03:06.246493 9817 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:03:06.671741 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 106: this message will repeat every 5th retry.
I20260430 20:03:07.794935 9854 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:03:07.795383 9854 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:03:07.796803 9854 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:03:07.799198 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:03:07.802719 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:03:07.803215 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:03:07.803488 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:03:07.804011 9854 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:03:10.716928 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:10.717363 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:03:13.195051 9943 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:13.195613 9943 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:03:13.197256 9943 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:03:13.198642 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:03:13.199136 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:03:13.199397 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:03:13.199920 9943 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:03:13.989398 9959 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:13.989915 9959 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:03:13.991998 9959 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:03:13.993484 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:03:13.994127 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:03:13.994489 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:03:13.995216 9959 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:03:15.720402 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:15.720757 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:03:15.806252 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 111: this message will repeat every 5th retry.
I20260430 20:03:20.489478 10073 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:03:20.490082 10073 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:03:20.491901 10073 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:03:20.495458 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:03:20.499262 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:03:20.499830 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:03:20.500258 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:03:20.500831 10073 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:03:20.723045 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:20.723495 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:03:22.964795 6400 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0dc0 after lost signal to thread 6271
W20260430 20:03:22.965692 6400 debug-util.cc:398] Leaking SignalData structure 0x7b08000bdde0 after lost signal to thread 6403
I20260430 20:03:25.382207 10146 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:25.382931 10146 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:03:25.388072 10146 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:03:25.389360 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:03:25.389860 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:03:25.390097 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:03:25.390635 10146 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:03:25.726177 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:25.726549 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:03:25.918773 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 116: this message will repeat every 5th retry.
I20260430 20:03:29.311295 10224 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:03:29.311935 10224 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:03:29.313939 10224 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:03:29.316424 6820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:03:29.319095 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:03:29.319458 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:03:29.319825 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:03:29.320401 10224 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:03:30.729212 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:30.729645 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:03:32.000646 6259 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (571 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:03:32.655025 10268 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:32.655529 10268 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:03:32.657079 10268 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:03:32.658674 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:03:32.659157 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:03:32.659377 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:03:32.659968 10268 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:03:32.738830 6404 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (463 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:03:32.791612 6111 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:35659 (494 consecutive failures): Network error: Failed to ping master at 127.3.190.62:35659: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:03:33.098693 10292 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:33.099210 10292 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:03:33.101212 10292 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:03:33.102698 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:03:33.103310 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:03:33.103611 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:03:33.105027 10292 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:03:35.591735 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 121: this message will repeat every 5th retry.
I20260430 20:03:35.732832 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:35.733242 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:03:36.425189 10332 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:36.425603 10332 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:03:36.427122 10332 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:03:36.428751 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:03:36.429280 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:03:36.429683 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:03:36.430336 10332 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:03:40.736531 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:40.737021 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:03:41.666169 10433 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:03:41.666831 10433 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:03:41.668550 10433 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:03:41.670641 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:03:41.673482 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:03:41.673885 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:03:41.674175 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:03:41.674803 10433 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 20:03:44.975756 6105 debug-util.cc:398] Leaking SignalData structure 0x7b08000422e0 after lost signal to thread 5960
W20260430 20:03:44.977243 6105 debug-util.cc:398] Leaking SignalData structure 0x7b08000cacc0 after lost signal to thread 6108
I20260430 20:03:45.740052 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:45.740455 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:03:46.193113 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 126: this message will repeat every 5th retry.
I20260430 20:03:46.784214 10524 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:46.784832 10524 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:03:46.786449 10524 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:03:46.787994 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:03:46.788551 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:03:46.788820 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:03:46.789333 10524 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:03:48.316638 10516 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:48.317203 10516 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:03:48.319447 10516 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:03:48.320891 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:03:48.321488 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:03:48.321758 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:03:48.322263 10516 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:03:50.743674 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:50.744084 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:03:53.648576 10638 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:03:53.649081 10638 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:03:53.651000 10638 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:03:53.653052 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:03:53.655791 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:03:53.656379 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:03:53.656644 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:03:53.657238 10638 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:03:55.746757 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:03:55.747105 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:03:56.710459 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 131: this message will repeat every 5th retry.
I20260430 20:03:59.207940 10731 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:03:59.208564 10731 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:03:59.211068 10731 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:03:59.212836 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:03:59.213395 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:03:59.213678 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:03:59.214210 10731 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:00.750859 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:00.751319 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:04:03.349848 10782 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:03.350473 10782 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:04:03.352445 10782 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:04:03.354120 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:04:03.354614 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:04:03.354854 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:04:03.355334 10782 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:04.457962 10820 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:04.458479 10820 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:04:04.460464 10820 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:04:04.461766 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:04:04.462306 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:04:04.462700 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:04:04.463301 10820 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:05.753981 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:05.754354 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:04:06.750267 10858 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:04:06.750778 10858 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:04:06.752359 10858 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:04:06.754571 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260430 20:04:06.757644 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:04:06.758267 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:04:06.758525 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:04:06.759141 10858 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 20:04:06.943493 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 136: this message will repeat every 5th retry.
I20260430 20:04:07.679522 10878 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:04:07.679960 10878 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:04:07.681541 10878 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:04:07.686350 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:04:07.686810 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:04:07.687179 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:04:07.687861 10878 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:04:08.204169 10888 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:08.204777 10888 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:04:08.206454 10888 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:04:08.208256 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:04:08.208784 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:04:08.209060 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:04:08.209771 10889 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:09.670729 10917 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:09.671662 10917 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:04:09.673400 10917 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:04:09.674937 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:04:09.675467 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:04:09.675724 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:04:09.676273 10917 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:10.758123 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:10.758555 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:04:13.022868 10968 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:13.023313 10968 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:04:13.025117 10968 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:04:13.026634 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:04:13.027324 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:04:13.027666 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:04:13.028266 10968 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:15.761250 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:15.761574 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:04:17.098994 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 141: this message will repeat every 5th retry.
I20260430 20:04:17.639422 11035 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:04:17.639993 11035 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:04:17.641534 11035 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:04:17.643738 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260430 20:04:17.646482 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:04:17.647233 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:04:17.647593 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:04:17.648309 11035 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:04:17.841267 11027 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:17.841962 11027 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:04:17.843628 11027 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:04:17.845064 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:04:17.845568 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:04:17.845834 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:04:17.846418 11027 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:18.250087 11045 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:04:18.250608 11045 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:04:18.252126 11045 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:04:18.256459 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
W20260430 20:04:18.256774 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
I20260430 20:04:18.257053 6820 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:04:18.257562 11045 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260430 20:04:20.301028 11084 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:20.301630 11084 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:04:20.303779 11084 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:04:20.305119 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:04:20.305684 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:04:20.305968 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:04:20.306452 11084 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:20.765295 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:20.765743 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:04:25.768689 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:25.769107 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:04:25.836997 11172 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:25.837487 11172 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:04:25.839085 11172 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:04:25.840529 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:04:25.840855 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:04:25.841135 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:04:25.841610 11172 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260430 20:04:26.618928 11175 raft_consensus.cc:493] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:26.619381 11175 raft_consensus.cc:515] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } }
I20260430 20:04:26.621476 11175 leader_election.cc:290] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977)
W20260430 20:04:26.622848 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
W20260430 20:04:26.623401 6149 leader_election.cc:343] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd13173a6c184acaa26b63bdbf8d6656 (127.3.190.3:45977): Not found: Tablet not found: fb30716bb9174641ba41885f792e6364
I20260430 20:04:26.623654 6149 leader_election.cc:304] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787275a4c5c64a6aa81be209814867bf; no voters: 499a4336efe645d4926c94600c732b82, cd13173a6c184acaa26b63bdbf8d6656
I20260430 20:04:26.624151 11175 raft_consensus.cc:2749] T fb30716bb9174641ba41885f792e6364 P 787275a4c5c64a6aa81be209814867bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:04:27.137348 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 146: this message will repeat every 5th retry.
W20260430 20:04:29.781437 6400 debug-util.cc:398] Leaking SignalData structure 0x7b08000407e0 after lost signal to thread 6271
W20260430 20:04:29.782876 6400 debug-util.cc:398] Leaking SignalData structure 0x7b08000d0740 after lost signal to thread 6403
I20260430 20:04:30.772718 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:30.773135 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:04:32.151734 6259 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (724 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:04:32.283056 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9960 after lost signal to thread 6770
W20260430 20:04:32.283766 6865 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8d20 after lost signal to thread 6868
I20260430 20:04:32.988164 11289 raft_consensus.cc:493] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260430 20:04:32.988760 11289 raft_consensus.cc:515] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499a4336efe645d4926c94600c732b82" member_type: VOTER last_known_addr { host: "127.3.190.1" port: 33861 } } peers { permanent_uuid: "787275a4c5c64a6aa81be209814867bf" member_type: VOTER last_known_addr { host: "127.3.190.2" port: 35955 } } peers { permanent_uuid: "cd13173a6c184acaa26b63bdbf8d6656" member_type: VOTER last_known_addr { host: "127.3.190.3" port: 45977 } }
I20260430 20:04:32.990381 11289 leader_election.cc:290] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861), 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955)
W20260430 20:04:32.992031 6294 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
W20260430 20:04:32.992643 6295 leader_election.cc:343] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787275a4c5c64a6aa81be209814867bf (127.3.190.2:35955): Not found: Tablet not found: d28c8f73e1bd4acd8e30805e34662440
I20260430 20:04:32.992928 6295 leader_election.cc:304] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd13173a6c184acaa26b63bdbf8d6656; no voters: 499a4336efe645d4926c94600c732b82, 787275a4c5c64a6aa81be209814867bf
I20260430 20:04:32.993496 11289 raft_consensus.cc:2749] T d28c8f73e1bd4acd8e30805e34662440 P cd13173a6c184acaa26b63bdbf8d6656 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260430 20:04:33.127131 6404 heartbeater.cc:646] Failed to heartbeat to 127.3.190.60:45507 (583 consecutive failures): Network error: Failed to ping master at 127.3.190.60:45507: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:04:33.205945 6111 heartbeater.cc:646] Failed to heartbeat to 127.3.190.62:35659 (614 consecutive failures): Network error: Failed to ping master at 127.3.190.62:35659: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:04:33.247598 11294 raft_consensus.cc:493] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 867d1f5512324df489edc5a61b79df54)
I20260430 20:04:33.248106 11294 raft_consensus.cc:515] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46e27ec79e44f329de14da21f60e60f" member_type: VOTER last_known_addr { host: "127.3.190.62" port: 35659 } } peers { permanent_uuid: "f2878444c3a64840b53f8d5f6fc6211d" member_type: VOTER last_known_addr { host: "127.3.190.61" port: 35031 } } peers { permanent_uuid: "867d1f5512324df489edc5a61b79df54" member_type: VOTER last_known_addr { host: "127.3.190.60" port: 45507 } }
I20260430 20:04:33.249784 11294 leader_election.cc:290] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659), 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507)
W20260430 20:04:33.251864 6818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260430 20:04:33.254664 6820 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 867d1f5512324df489edc5a61b79df54 (127.3.190.60:45507): Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:04:33.255105 6818 leader_election.cc:336] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f46e27ec79e44f329de14da21f60e60f (127.3.190.62:35659): Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111)
I20260430 20:04:33.255429 6818 leader_election.cc:304] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2878444c3a64840b53f8d5f6fc6211d; no voters: 867d1f5512324df489edc5a61b79df54, f46e27ec79e44f329de14da21f60e60f
I20260430 20:04:33.255944 11294 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f2878444c3a64840b53f8d5f6fc6211d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260430 20:04:33.548682 6255 debug-util.cc:398] Leaking SignalData structure 0x7b08000cbe80 after lost signal to thread 6124
W20260430 20:04:33.549615 6255 debug-util.cc:398] Leaking SignalData structure 0x7b08000de220 after lost signal to thread 6258
I20260430 20:04:35.790505 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:35.790926 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260430 20:04:37.199536 6149 consensus_peers.cc:597] T 8da8b25a7d17492b8830605a61a2f2ea P 787275a4c5c64a6aa81be209814867bf -> Peer 499a4336efe645d4926c94600c732b82 (127.3.190.1:33861): Couldn't send request to peer 499a4336efe645d4926c94600c732b82. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da8b25a7d17492b8830605a61a2f2ea. This is attempt 151: this message will repeat every 5th retry.
I20260430 20:04:40.794305 6769 master_runner.cc:186] Error getting master registration for 127.3.190.62:35659: Network error: Client connection negotiation failed: client connection to 127.3.190.62:35659: connect: Connection refused (error 111),
I20260430 20:04:40.794865 6769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260430 20:04:41.233238 6823 maintenance_manager.cc:419] P f2878444c3a64840b53f8d5f6fc6211d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1203 bytes on disk
I20260430 20:04:41.244505 6822 maintenance_manager.cc:643] P f2878444c3a64840b53f8d5f6fc6211d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.006s sys 0.004s Metrics: {"dirs.queue_time_us":779,"thread_start_us":874,"threads_started":2,"wal-append.queue_time_us":775}
W20260430 20:04:41.809196 6414 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:04:41.809645 6414 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:04:41.809970 6414 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
W20260430 20:04:41.810254 6414 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
F20260430 20:04:41.822659 6423 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b2b7ae30e3404fe8be26c7cc54e7d102 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1777579481 (unix time) try "date -d @1777579481" if you are using GNU date ***
F20260430 20:04:41.824133 6428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.190.62:35659,127.3.190.61:35031,127.3.190.60:45507, num_attempts: 767) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.3.190.60:45507: connect: Connection refused (error 111)
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000ef8) received by PID 3832 (TID 0x7fdbc2ffd700) from PID 3832; stack trace: ***
@ 0x2de460 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2e0943 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7fdbd508d980 (unknown) at ??:0
@ 0x7fdbd024ee87 gsignal at ??:0
@ 0x7fdbd02507f1 abort at ??:0
@ 0x2e4456 abort at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7fdbd32e6da4 google::LogMessage::SendToLog() at ??:0
@ 0x7fdbd32e7950 google::LogMessage::Flush() at ??:0
@ 0x7fdbd32eca8b google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x360594 kudu::MasterStressTest::CreateTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x38e45e _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3846
@ 0x38e311 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x38da5b std::__1::__thread_proxy<>() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2de320 __tsan_thread_start_func at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7fdbd50826db start_thread at ??:0
@ 0x7fdbd033161f clone at ??:0